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Abstract. This article analyzes the recovery performance of two popular finite 
dimensional approximations of the sparse spikes deconvolution problem over Radon 
measures. We examine in a unified framework both the regularization (often 
referred to as Lasso or Basis-Pursuit) and the Continuous Basis-Pursuit (C-BP) 
methods. The Lasso is the de-facto standard for the sparse regularization of inverse 
problems in imaging. It performs a nearest neighbor interpolation of the spikes 
locations on the sampling grid. The C-BP method, introduced by Ekanadham, 
Tranchina and Simoncelli, uses a linear interpolation of the locations to perform a 
better approximation of the infinite-dimensional optimization problem, for positive 
measures. We show that, in the small noise regime, both methods estimate twice the 
number of spikes as the number of original spikes. Indeed, we show that they both 
detect two neighboring spikes around the locations of an original spikes. These results 
for deconvolution problems are based on an abstract analysis of the so-called extended 
support of the solutions of £^-type problems (including as special cases the Lasso 
and C-BP for deconvolution), which are of an independent interest. They precisely 
characterize the support of the solutions when the noise is small and the regularization 
parameter is selected accordingly. We illustrate these findings to analyze for the 
first time the support instability of compressed sensing recovery when the number 
of measurements is below the critical limit (well documented in the literature) where 
the support is provably stable. 


1. Introduction 

We consider the problem of estimating an nnknown Radon measnre mo G AI(T) 
from low-resolntion noisy observations 


y = <h(mo) + w G L^(T) 


( 1 ) 


where w G L^(T) is some measnrement noise, and <I> : AI(T) —)■ L^(T) is an integral 
transform with smooth kernel (p G (^^(T x T), i.e. 
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A typical example of such an operation is the convolution, where ip{x, y) = (p{x — y) for 
some smooth function (p dehned on the torus T = M/Z (i.e. an interval with periodic 
boundary conditions). We focus our attention here for simplicity on the compact 1-D 
domain T, but the algorithms considered (Lasso and C-BP) as well as our theoretical 
analysis can be extended to higher dimensional settings (see Section 1.4). 

1.1. Sparse Regularization 

The problem of inverting (1) is severely ill-posed. A particular example is when 
$ is a low pass filter, which is a typical setting for many problems in imaging. In 
several applications, it makes sense to impose some sparsity assumption on the data to 
recover. This idea has been introduced hrst in the geoseismic literature, to model 
the layered structure of the underground using sparse sums of Dirac masses [13]. 
Sparse regularization has later been studied by David Donoho and co-workers, see for 
instance [17]. 

In order to recover sparse measures (i.e. sums of Diracs), it makes sense to consider 
the following regularization 

~ <^>(Hir + A|m|(T) (3) 

m£M{T) 2 

where |m|(T) is the total variation of the measure m, dehned as 

|m|(T) =■ sup I j ^jJ{x)dm{x) ; ^ G C(T), ||^7||oo ^ l| . (4) 

This formulation of the recovery of sparse Radon measures has recently received lots of 
attention in the literature, see for instance the works of [6, 15, 10]. 

1.2. Lasso 

The optimization problem (3) is convex but inhnite dimensional, and while there 
exists solvers when <I> is measuring a hnite number of Fourier frequency (see [10]), 
they do not scale well with the number of frequencies. Furthermore, the case of an 
arbitrary linear operator <F is still difficult to handle, see [6] for an iterative scheme. 
The vast majority of practitioners thus approximate (3) by a hnite dimensional problem 
computed over a hnite grid Q '=' {zi ; i G |0, P — 1]} C T, by restricting their attention 
to measures of the form 

p-i 

ma,g = Y,aA.eM(T). 

i=0 

For such a discrete measure, one has |m|(T) = |ai| = ||a||i, which can be 

interpreted as the fact that | ■ |(T) is the natural extension of the G norm from 
hnite dimensional vectors to the inhnite dimensional space of measures. Inserting this 
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parametrization in (3) leads to the celebrated Basis-Pursuit problem [12], which is also 
known as the Lasso method in statistics [33], 


min + A||a|| 

aGR^ 2 

where in the following we make use of the notations 


(5) 


p-i 


<hga =■ ^ Zi), 

i=0 
p-i 

$^6 =■ ^ bid2(p{-, Zi), 


( 6 ) 

(7) 


i=0 


and d 2 <p is the derivative with respect to the second variable. One can understand (5) 
as performing a nearest neighbor interpolation of the Dirac’s locations. 

This approximation is however quite crude, and we recently showed in [21] that it 
leads to imperfect estimation of both the number of spikes and their locations. Indeed, 
this problem typically recovers up to twice as many spikes as the input measures, because 
spikes of mo gets duplicated as the two nearest neighbors on the grid Q. 

Note that while we focus in this paper on convex recovery method, and in particular 
£^-type regularization, there is a vast literature on the subject, which makes use of 
alternative algorithms, see for instance [28, 5] and the references therein. 

1.3. Continuous Basis-Pursuit (C-BP) 

To obtain a better approximation of the inhnite dimensional problem, [23] proposes 
to perform a hrst order approximation of the kernel. This method assumes that the 
unknown measure is positive. To ease the exposition, we consider a uniform grid 
g Id' \ijp ^ g |0^ p — IJI of P points, so that the grid size is h =' 1/P. The C- 
BP method of [23] solves 


min -||■u 

(a,b)GR^xR^" 2 


<hga — -|-A||a||i subject to 


,,, h 

\b\ ^ 2 


( 8 ) 


where the inequality should be understood component-wise. Note also that the obtained 
a is always nonnegative, hence the C-BP method is tailored for the recovery of positive 
measures. This is a convex optimization problem, which can be solved using traditional 
conic optimization methods. As detailed in Section 4.2, this problem can also be re-cast 
as a Lasso in dimension 2P with positivity constraints (see Section 4.2). Hence it can 
be solved using a large variety of first order proximal method, the most simple one being 
the Forward-Backward, see [3] and the references therein. 

If (a*,P) are solutions of (8), one recovers an output discrete measure dehned by 


m* = 




where x] 


a *^0 


def. . , bi 

= ih+—, 
a* 


(9) 
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where we set ^ = 0 whenever a* = 0. The rationale behind (8) is to perform a first order 

Taylor approximation of the operator <h, where the variable r* =' bi/oi G [—h/2,h/2] 
encodes the horizontal shift of the Dirac location with respect to the grid sample ih. 
The landmark idea introduced in [23] is that, while the optimization is non-convex with 
respect to the pair (a, r), it is convex with respect to the pair (a, 6). 

1 . 4 . Extensions 

While we restrict here the exposition to 1-D problems, the C-BP formulation (8) 
can be extended to cope with measures in arbitrary dimension d ^ 1, i.e. to consider 
mo G (¥“'). This requires to dehne at each sampling grid point indexed by f a vector 
bi = {bi^k)t=i ^ together with the constraint ||&j||oo ^ | a-i, and also to use a matrix 
dehned as 

d 

= G L2(T'') 

i^g k=l 

where denote the differential operator with respect to the direction in W^. Our 
analysis carries over to this setting without major difficulties. 

The paper [23] also proposes other interpolation schemes than a hrst order Taylor 
expansion at the grid points. In particular, they develop a “polar” interpolation which 
makes use of two adjacent grid points. This method seems to outperform the linear 
interpolation in practice, and has been employed to perform spikes sorting in neuronal 
recordings [22]. 

Extending the results we propose in the present paper to these higher dimensional 
settings and alternative interpolation schemes is an interesting avenue for future work. 

Let us also mention that an important problem is to extend the C-BP method (9) to 
measures with arbitrary signs and that can even be complex-valued. Unfortunately, the 
corresponding constraint |6| ^ |a| is then non-convex, which makes the mathematical 
analysis apparently much more involved. A non-convex and non-smooth optimization 
solver is proposed for this problem in [25], and shows promising practical performance 
for spectrum estimation. 

1.5. Previous Works 

Most of the early work to assess the performance of convex sparse regularization has 
focussed its attention on the hnite dimensional case, thus considering only the Lasso 
problem (5). While the literature on this subject is enormous, only very few works 
actually deal with deterministic and highly correlated linear operators such as low-pass 
convolution kernels. The initial works of Donoho [17] study the Lipschitz behavior of 
the inverse map y 1 —)■ a*, where a* is a solution of (5), as a function of the bandwidth 
of the bandpass hlter. The hrst work to address the question of spikes identihcation 
(i.e. recovery of the exact location of the spikes over a discrete grid) is [19]. This work 
uses the analysis of d regularization introduced by Fuchs in [26]. This type of analysis 



Sparse Spikes Deeonvolution on Thin Grids 


5 


ensures that the support of the input measure is stable under small noise perturbation of 
the measurements. Our hnding is that this is however never the case (support is always 
unstable) when the grid is thin enough, and we thus introduce the notion of “extended 
support”, which is in some sense the smallest extension of the support which is stable. 
The idea of extending the support to study the recovery performance of methods can 
be found in the work of Dossal [18] who focusses on noiseless recovery and stability in 
term of error. 

Recently, a few works have studied the theoretical properties of the recovery over 
measures (3). Candes and Fernandez-Granda show in [10] that this convex program 
does recover exactly the initial sparse measure when tc = 0 and A —)■ 0, if the spikes are 
well-separated. The robustness to noisy measurements is analyzed by the same authors 
in [9] using an Hilbertian norm, and in [24, 2] in terms of spikes localization. The 
work of [31] analyzes the reconstruction error. Lastly, [21] provides a condition ensuring 
that (3) recovers the same number of spikes as the input measure and that the error in 
terms of spikes localization and elevation has the same order as the noise level. 

Very few works have tried to bridge the gap between these grid-free methods over 
the space of measures, and hnite dimensional discrete approximations that are used 
by practitioners. The convergence (in the sense of measures) of the solutions of the 
discrete problem toward to ones of the grid-free problem is shown in [32] , where a speed 
of convergence is shown using tools from semi-inhnite programming [30]. The same 
authors show in [4] that the discretized problem achieves a similar prediction error 
as the grid-free method. In [21], we have shown that solutions of the discrete Lasso 
problem estimate in general as much as twice the number of spikes as the input measure. 
We detail in the following section how the present work gives a much more precise and 
general analysis of this phenomenon. 

1.6. Contributions 

Our hrst contribution is an improvement over the known analysis of the Lasso in an 
abstract setting (that is (5) when is replaced with any linear operator —)■ L^(R)). 
Whereas Fuchs’ result [26] characterizes the support recovery of the Lasso at low noise, 
our previous work [21] has pointed out that when Fuchs’criterion is not satished, the 
nonzero components of the solutions of the Basis-Pursuit at low noise are contained in 
the extended support, that is the saturation set of some minimal norm dual eertificate. 
In this work, we provide a characterization of this minimal norm certificate, and we give 
a sufficient condition which holds generically and which ensures that all the components 
of the extended support are actually nonzero (with a prediction on the signs). Our main 
result in this direction is Theorem 1. 

Our second contribution applies this result to Problem (5) on thin grids. After 
recalling the convergence properties of Problem (5) towards (3), we show that under 
some assumption, if the input measure mo = mao,xo = c>:o,udxo^,, has support on 

the grid (he. Xq^^, G Q for all u), the model at low noise actually reconstructs pairs of 
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N 

^ (ttA,1/5x0,. + /^A,1/5x0,, where G {-1, +1}, (10) 

i/=i 

and sign(aA,i/) = sign(/3A,,/) = sign(ao,!/)- (11) 

The precise statement of this result can be found in Theorem 2. Compared to [21] 
where it is predicted that spikes could appear at most in pairs, this result states that 
all the pairs do appear, and it provides a closed-form expression for the shift e. That 
closed-form expression does not vary as the grid is rehned, so that the side on which 
each neighboring spike appears is in fact intrinsic to the measure, we call it the natural 
shift. Moreover, we characterize the low noise regime as = 0(1) and A = 0{h). 

Then, we turn to the Continuous Basis-Pursuit (8). We hrst study this problem 
in an abstract setting, where it is reformulated as a Lasso with positivity constraints. 
We derive similar noise robustness properties as for the Lasso, and we characterize the 
extended support, see Theorem 3. Working on a thin grid, we show the T-convergence 
of Problem (8) towards the Beurling Lasso (3) with positivity constraints and we give 
a hne analysis of the support of the solutions as the grid stepsize tends to zero. We 
also stndy the low noise behavior when the measnre has support on the grid: under a 
suitable assumption, the recovered spikes appear again in pairs, 

N 

mA = ^ (caa ,1/5x0,.-i-t. + /9 a, !/5xo,.+e. h/ 2 ) where 

V=1 

see Theorem 4. A closed form expression for e is given, which depends on some 
corresponding natural shift intrinsic to the measnre (which differs from the one of the 
Lasso). The corresponding low noise regime is characterized by = 0(1) and 
A = 0{h^). 

It is important to realize that, in this setting of convolution on thin grids, our 
contributions give important information abont the strncture of the recovered spikes 
when the noise w is small. This is especially important since, on contrary to common 
belief, the spikes locations for Lasso and C-BP are not stable: even for an arbitrary 
small noise w, neither methods retrieve the correct inpnt spikes locations. 

Eventually, we illustrate in Section 6 these theoretical results with numerical 
experiments. We hrst display the evolntion of the solution path A 1 —)■ oa (a solution 
of (5)) and A i-A {a\,b\) (a solution of (8)). These paths are piecewise-affine, and 
our contributions (Theorems 2 and 4) precisely characterize the hrst affine segment of 
these paths, which perfectly matches the nnmerical observations. We then illustrate our 
abstract analysis of the Lasso problem (5) (as provided by Theorem 1) to characterize 
nnmerically the behavior of the Lasso for compressed sensing (CS) recovery (i.e. when 
one replaces the hltering appearing in (5) with a random matrix). The literatnre 
on CS only describes the regime where enough measurements are available so that the 
snpport is stable, or does not study support stability but rather G stability. Theorem 1 


Eu e {—1, -1-1}, 
-h/2 <tu < h/2, 
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allows us to characterize numerically how much the support becomes unstable (in 
the sense that the extended support’s size increases) as the number of measurements 
decreases (or equivalently the sparsity increases). 


1.7. Notations and preliminaries 


The set of Radon measures (resp. positive Radon measures) is denoted by A^(T) 
(resp. ■*■(¥)). Endowed with the total variation norm (4), A1(T) is a Banach space. 
Another useful topology on A1(T) is the weak* topology: a sequence of measures 
(m„)neN weak* converges towards m G A4(T) if and only if for all tjj G C(T), 
hm„^+ooAny bounded subset of A1(T) (for the total variation) is 
relatively sequentially compact for the weak* topology. Moreover the topology induced 
by the total variation is stronger than the weak* topology, and the total variation is 
sequentially lower semi-continuous for the weak* topology. Throughout the paper, given 
a G and xq G T^, the notation ma,xo = hints that 0 for all z/ 

(contrary to the notation ma^g), and that the are pairwise distinct. 

The properties of <h : A4(T) —)■ L^(T) and its adjoint are recalled in Proposition 6 
in Appendix. The cxd, 2-operator norm of <h* : L^(T) —)■ C'(T) is dehned as ||*h*||oo ,2 = 
sup {||<h*tc||oo ; w G T^(T), ||tc||L 2 ^ 1} (and the oo, 2 operator norm of a matrix is dehned 
similarly). Given a vector xq G $3,0 refers to the linear operator —)■ L^(T), with 

N 

Va G <ha:oa =' ^ auip{-,xo,u)- 

i/=i 

It may also be seen as the restriction of <h to measures supported on the set 
{xq^i, ; z/ G |1, iV]}. A similar notation is adopted for (replacing ip(-,xo^i,) with 

d 2 ^{-, ,a;o,!y). The concatenation of and is denoted by T^jq =' ^$ 3,0 

We shall rely on the notion of set convergence. Given a sequence (Gn)neN of subsets 
of T, we dehne 


limsupCn = < a; G T ; liminf (i(a;, G„) 

n—)-+oo 1^ n—)-+oo 


lim inf = < a; G T ; lim sup d{x, Cn) 

n—>-+oo 1^ )-+oo 



( 12 ) 

(13) 


where d is dehned by d{x, C) = inix'^c |a;' — a;| and |a; — a;'| refers to the distance between 
X and x' on the torus. If both sets are equal, let C be the corresponding set (then C is 
necessarily closed), we write 


lim Cn = C. (14) 

n—>-+00 

If the sequence (G„)„eN is nondecreasing (G„ C G„+i), then hm„_j.ooC'„ = UneN^^’ 
if it is nonincreasing {Cn D Cn+i) then lim„_j.ooGn = nneN^»^ (where C denotes the 
closure of C). We refer the reader to [29] for more detail about set convergence. We 
shall also use this notion in Hilbert spaces, with obvious adaptations. 
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The aim of this section is to study the low noise regime of the Lasso problem in an 
abstract hnite dimensional setting, regardless of the grid stepsize. In this framework, the 
columns of the (hnite dimensional) degradation operator need not be the samples of a 
continuous {e.g. convolution) operator, and the provided analysis holds for any general 
Lasso problem. We extend the initial study of Fuchs of the basis pursuit method 
(see [26]) which gives the analytical expression of the solution when the noise is low 
and the support is stable. Here, provided we have access to a particular dual vector rjQ, 
we give an explicit parametrization the solutions of the basis pursuit at low noise even 
when the support is not stable. This is especially relevant for the deconvolution problem 
since the support is not stable when the grid is thin enough. 

2.1. Notations and optimality conditions 

We consider in this section observations in an arbitrary Hilbert space TL, which 
might be for instance L^(T) (as in the previous section) or a hnite dimensional vector 
space. The linear degradation operator is then denoted as A : —)■ "H. Let us 

emphasize that in this section, for a E ||a||oo = niaxo^fc^p_i \ak\. 

Given an observation yo = Aoq E TL {ot y = yo + w, where w E TL), we aim at 

reconstructing the vector oq G by solving the Lasso problem for A > 0, 

min hy - Aaf + A||a||i {Vx{y)) 

aeRP 2 

and for A = 0 we consider the (Basis-Pursuit) problem 

min ||a||i such that Aa = yo. (^o(l/o)) 

a&RP 

If a G M^, we denote by /(a), or / when the context is clear, the support of a, 

i.e. I{a) =' {i G |0, P — 1] ; a* 7 ^ 0}. Also, we let sj =' sign(a/), and supp^(a) =' 

{(i,Si) ; i E 1} the signed support of a. 

The optimality conditions for Problems (Vx{y)) and (Po(l/o)) are quite standard, 
as detailed in the following proposition. 

Proposition 1. Let y eTL, and ax E M^. Then ax is a solution to (Vx{y)) if and only 
if there exists px ETC such that 

||A*pa||oo ^ 1, and {A*px)i = sign(aA,/), (15) 

\A*px + A*{Aax-y) = t). (16) 

Similarly, if ao E then oq is a solution to (Po(l/o)) if and only if Aao = yo and 
there exists p eTL such that. 


||^*p||oo ^ 1 and {A*p)i = sign(ao,/). 


(17) 
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Conditions (15) and (17) merely express the fact that px = (resp. p =' A*p) 
is in the subdifferential of the £^-norm at ax (resp. oq). In that case we say that px 
(resp. p) is a dual certificate for ax (resp. Oq). Condition (17) is also called the source 
condition in the literature [7]. 

The term dual certificate stems from the fact that px (resp. p) is a solution to the 
dual problem to {Vx{y)) (resp. (Po(l/o))), 


. r y ^ 

inf w - P , 

pec A 2 

CPxit/)) 

resp. sup(|/o,p), 
pec 

mm)) 

where C =' < p G "H ; max \{A*p)k\ ^ 1 i . 

i fcG[0,P-l]" J 

(18) 


If a is a solution to (Vx{y)) and px is a solution to (Vx{y)), then (15) and (16) hold. 
Conversely, for any a G and any px G TL, if (15) and (16) hold, then a is a solution 
to (Vx{y)) and px is a solution to (Vx{y)). A similar equivalence hold for (Po(l/o)) 
and {Vo{yo)). 

Remark 1. In general, the solutions to (Vx{y)) and (Po(l/o)) need not be unique. 
However, the dual certihcate px = A*px which appears in (15) and (16) is unique. 
On the contrary, the dual certihcate p = A*p which appears in (17) is not unique in 
general. 

We say that a vector oq is identifiable if it is the unique solution to (Po(l/o)) for the 
input y = Aoq. The following classical result gives a sufficient condition for oq to be 
identihable. 

Proposition 2. Let oq G such that Aj is injective and that there exists p eTL such 
that 

||(A*p)7c||oo < 1 and {A*p)i = siga.{aoj), (19) 

where /'^ = |1, P] \ / . Then Oq is identifiable. 

Conversely, if oq is identihable, there exists p G L^(T) such that (19) holds and Aj 
is injective (see [27, Lemma 4.5]). 

2.2. Extended support of the Lasso 

From now on, we assume that the vector oq G is identihable {i.e. Oq is the unique 
solution to (Vo{yo)) where yo = Aafij. We denote by / = supp(ao) and sj = sign(ao,/) 
the support and the sign of Oq. 

It is well known that (Po(l/o)) is the limit of {Vxiy)) for A —)■ 0 (see [12] for the 
noiseless case and [27] when the observation is y = y^ + w and the noise w tends to zero 
as a multiple of A) at least in terms of the convergence. In terms of the support of 
the solutions, the study in [21], which extends the one by Fuchs [26], emphasizes the 
role of a specihc minimal-norm certificate po which governs the behavior of the model 
at low noise regimes. 
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Definition 1 (Minimal-norm certificate and extended support). Let Qq G and let 
Po be the solution to (Vq^i/q)) with minimal L'^-norm. The minimal-norm certificate of 
oq is defined as po = ^*Po- The set of indices ext(ao) =' {1 ^ ^ P ; \{po)j\ = 1} is 

called the extended support of Oq, and the set ext^(ao) =' {{j, {vo)j) ; j G ext(ao)} C 
[0, P - 11 X {—1,1} is called the extended signed support of ao. 

Remark 2. In the case where oq is a solution to (Po(l/o)) (which is the case here since we 
assume that Oq is an identifiable vector for (Po(l/o))); we have (J, sign(ao,/)) C ext^(ao). 
The minimal norm certificate thus turns out to be 

? 7 o = A*pq where po = argmin {||p ||2 ; ||-4*p||oo ^ 1 and A*jp = si] . ( 20 ) 

p&'H 

It is shown in [21] that there exists a low noise regime where the (signed) support 
of any solution d\ of Pa(|/o + is included in ext^(ao), supp^ oa C ext^(ao). It is 
therefore crucial to understand precisely the behavior of po and the structure of the 
extended (signed) support ext^(ao). The following (new) result gives a characterization 
of Pq. 

Lemma 1. Let {J,sj) C |0, P — 1] x { — 1,1} such that (J, sign((ao)/)) C {J,sj) and 
Aj has full rank. Define vj = {A*jAj)~^sj. 

Then {J,sj) is the extended signed support of Qq, i.e. {J,sj) = ext^(ao), if and 
only if the following two conditions hold: 

• for all jEJ\I,Vj = 0 or Sj = — sign(vj), 

• \\A*j.AjVj\\oc, < 1 . 

In that case, the minimal norm certificate is given by po = A*A'^’*sj. 

Proof. Writing the optimality conditions for (20), we see that p G L^(T) is equal to po if 
and only if ||A*p||oo ^ 1, A*jp = sign(aoy), and there exists G (R'*')'^ and G (R'*')^ 
such that: 


2p -\- Au+ — Au_ = 0, (21) 

where for i G D, (resp. M-,i) is a Lagrange multiplier for the constraint 

{A*p)i ^ 1 (resp. {A*p)i ^ —1) which satisfies the complementary slackness condition: 
u+fi{A*p)i — 1) = 0 (resp u_fi{A*p)i -|- 1) = 0), and for i E I, («+,* — «_,*) is the 
Lagrange multiplier for the constraint {A*p)i = sign(ao)i. 

Now, let {J,sj) = ext^(ao) (so that J determines the set of active constraints) and 
p = Po- Using the complementary slackness condition we may reformulate (21) as 

Po - Ajvj = 0, 

for some v G R"^, where Vj = 0 or signvj = —{A*po)j for j G J \ I, and Vj = 0 for 
j G |0, P — 1] \ J. Inverting this relation, we obtain vj = {AjAj)~^{pq)j, and the stated 
conditions hold. 
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Conversely, let {J,sj) C |0, P — 1] x {—1,1} (not necessarily equal to ext=*=(ao)) 
such that (/, sign((ao)/)) C {J,sj) and that the conditions of the lemma hold, with 
vj = {A*jAj)~^sj. Then, setting p = -AjVj, we see that ||A*j9||oo ^ 1, A}p = sign(ao)/, 
and (21) holds with the complementary slackness when setting = |max(nj,0), 
U-j = I max(—Uj, 0) for j E J and u±j = 0 for j ^ J. Then p = po and the equivalence 
is proved. □ 


As mentioned above, the minimal norm certihcate governs the (signed) support of 
the solution at low noise regimes insofar as the latter is contained in the extended signed 
support. The following theorem shows that, in the generic case, both signed supports 
are equal. 

Theorem 1. Let Oq G \ {0} be an identifiable signal, J =' ext(ao) such that Aj has 
full rank, and vj =' {A*jAj)~^ sign(77o,j)- Assume that for all j E J \ I, Vj 7^ 0. Then, 
there exists constants > 0, ^(2) > 0 (which depend only on A, I and sign(ao,/) j 

such that for A ^ ( min |ao/| ) and all w E Ti with ||tc|| ^ C'^^^A the solution d\ 

\iGl ' ) 

of {V\{y)) is unique, supp(aA) = J and it reads 

d\,j = ao,j + A(jW - \{A*jAj)~^ sign(r7o,j), 
where A] = {A*jAj)-^A*j. 


Proof. We dehne a candidate solution d by 

d.j = ao,j + AjW - Xvj, djc = 0 

and we prove that a is the unique solution to (Vx{yo + w)) using the optimality 
conditions (15) and (16). 

We hrst exhibit a condition for sign(aj) = sign(77o,j)- To shorten the notation, we 
write sj = sign(77o,j)- Since for i E I, ao,i 0, the constraint sign(a/) = s/ is implied 
by 


||oo,2||'a^|| + 


3A < T, where T = min |ao,/| > 0, 

ie/ 


and Rj : u hE ui is the restriction operator. As for if = J \ I, for all k E K ao,fc = 0 
but we know from Lemma 1 that sign(nfc) = —Sk- The constraint sign(ax) = sk is thus 
implied by 

||^x^j||oo,2|kll ^ A (min \vk\ ) . 

\k£K J 
^ >0 ^ 

Hence, we have signcij = sign770,j = sj, and by construction supp(a) = J with 

A*j{y-Ad) = \sj. (22) 

To ensure that a is the unique solution to (Vx{y)) with y = yQ + w, it remains to 
check that 


MJ.fe - .4a)||co < A. 


(23) 
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Since we have 


y- Ad = w- Aj{A*jAj) ^AjW + \Aj{A*jAj) ^sj = P],er(A*)W + AAj’*sj, 
we see that (23) is implied by 


II ^ Jc-fker(Aj) II 2,00 IIII A(1 ||hO,J“ ||oo) ^ 0 


where by construction ||?7o,jc||oo < 1- 

Putting everything together, one sees that d is the unique solution of (V\{y)) if the 
following affine inequalities hold simultaneously 


Cl 


C4\\W 


+ C 2 X < T where 
I tel ^ C3A where 
— C5A < 0 where 


Cl = ||i?M+|U,2, 

, = ||n,|U, 

C3 = {\\Rk A J Woo,2) 

C4 = ||2l}.Pker(A}) 
C5 =■ 1 - ||? 70 ,J=||oo 


^ ( min \vk\ ) 
\keK J 

||2,007 

> 0 . 


> 0 , 


(24) 

(25) 

(26) 


Hence, for ||t(;|| < min(c3, ^)A and + C2j A < T, the hrst order optimality 

conditions hold. □ 


Remark 3 (Comparison with the analysis of Fuchs). When J = I, Theorem 1 recovers 
exactly the result of Fuchs [26]. Note that this result has been extended beyond the 
R setting,see in particular [35, 34] for a unihed treatment of arbitrary partly smooth 
convex regularizers. For this result to hold, i.e. to obtain J = J, one needs to impose 
that the following pre-certiheate 

riF A*Ap*sj (27) 

is a valid certiheate, i.e. one needs that ||77 f, 7 '=||oo < 1- This condition is often called the 
irrepresentahility condition in the statistics literature (see for instance [37]). It implies 
that the support I is stable for small noise. Unfortunately, it is easy to verify that for the 
deconvolution problem, in general, this condition does not hold when the grid stepsize 
is small enough (see [21, Section 5.3]), so that one cannot use the initial result. This 
motivates our additional study of the extended support ext(ao) D /, which is always 
stable to small noise. While this new result is certainly very intuitive, to the best of 
our knowledge, it is the hrst time it is stated and proved, with explicit values of the 
stability constant involved. 

Remark 4. Theorem 1 guarantees that the support of the reconstructed signal d\ at low 
noise is equal to the extended support. The required condition Vj 7^ 0 in Theorem 1 is 
tight in the sense that if Vj = 0 for some j E J\I, then the saturation point of px may 
be strictly included in J. Indeed, it is possible, using similar calculations as above, to 
construct w such that suppciA C J with A and ||tc||2/A arbitrarily small. 
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In this section, we focus on inverse problems with smooth kernels, such as for 
instance the deconvolution problem. Our aim is to recover a measure tuq G A1(T) from 
the observation = <hmo oi y = <hmo + w, where (p G C^{T x T) (/c ^ 2), w G LS{T) 
and 

Vx G T, (<l>m)(a;) =■ / (p{x,y)(\.m{y), (28) 

Jf 

so that <h : A1(T) —)■ L^(T) is a bounded linear operator. Observe that <h is in fact 
weak* to weak continuous and its adjoint is compact (see Lemma 6 in Appendix). 

Typically, we assume that the unknown measure mo is sparse, in the sense that it 
is of the form mo = oiQ^uXo^u for some N G N*, here q;o,i/ G M* and the G T are 
pairwise distinct. 

The hrst approach we study is the one of the (discrete) Basis Pursuit. We look for 
measures that have support on a certain discrete grid ^ C T, and we want to recover the 
original signal by solving an instance of (Poido)) or {V\{y)) on that grid. Specihcally, 
we aim at analyzing the behavior of the solutions at low noise regimes (he. when the 
noise w is small and A well chosen) as the grid gets thinner and thinner. To this end, we 
take advantage of the characterizations given in Section 2 with TL =' L^(T), regardless 
of the grid, and we use the Beurling Lasso (3) as a limit of the discrete models. 

3.1. Notations and preliminaries 

For the sake of simplicity we only study uniform grids, i.e. Q =' 
{ih ; i G |0, P — 1]} where h is the stepsize. Moreover, we shall consider sequences 
of grids (^n)nGN such that the stepsize vanishes (h^ = -^—)■ 0 as n —)■ +cxd) and to ensure 
monotonicity, we assume that Qn+i C Qn- For instance, the reader may think of a dyadic 
grid {i.e. hn = ^). We shall identify in an obvious way measures with support in Qn 
{i.e. of the form o-k^khn) and vectors a G 

The problem we consider is a particular instance of {V\{y)) (or (Po(|/o))) when 
choosing A as the restriction of $ to measures with support in the grid 

A " = (43(-,0),.. (P - Ijfcj) . (29) 

More explicitely, on the grid Qn, we solve 

min h\y - + M\a\\i, {PZiv)) 

aGR^" 2 

and min ||a||i such that = yo. {Po{yo)) 

aGRPri 

We say that a measure mo = (^o,u^xo,y (with ^ 0 and the Xo^y^s pairwise 

distinct) is identihable through (Po(?/o)) if h can be written as mo = '^kZo^ ca^ihn and 
that the vector a is identifiable using (Pq (?/o))- 
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As before, given a G we shall write J(a) =' {i G |0, — 1 ] ; Oj 7 ^ 0} or simply 

I when the context is clear. 

The optimality conditions (16) amount to the existence of some p\ G L^(T) such 

that 

max |(<l>*pA)(/i:h„)| ^ 1, and (<h*pA)(-fh„) = sign(aA,/), (30) 

O^fcsgPn —1 

A<I>*pa + *h*($aA - y) = 0. (31) 

Similarly the optimality condition (17) is equivalent to the existence of p G L^(T) such 
that 


max |(<h*p)(/ch„)| ^ 1 and (<h*p)(Jh„) = sign(ao,/). (32) 

— 1 

Notice that the dual certihcates are naturally given by the sampling of continuous 
functions f] = <h*p : T —)■ M, and that the notation r]{Ihn) or (<l>*p)(/h„) stands for 
{r]{ihn))i£i where / = /(oq) (and similarly for rjx = ^*px and /(oa)). 

If mo is identihable through (Voivo)), the minimal norm certihcate for the 
problem {V^iyo)) (see Section 2) is denoted by ?7g, whereas the extended support on 
is dehned as 


extnmo = {t G ; rj^it) = ±1}. (33) 

From Section 2, we know that the extended support is the support of the solutions at 
low noise. 

3.2. The limit problem: the Beurling lasso 

It turns out that Problems iVxiy)) and (Pq (l/o)) have natural limits when the grid 
gets thin. Embedding those problems into the space A4(T) of Radon measures, the 
present authors have studied in [21] their convergence towards the Beurling-LASSO used 
in [15, 10, 6, 31]. 

The idea is to recover the measure mo using the following variants of {Vx{y)) 
and {Voipjo))- 

min ^||p-<l>mp + A|m|(T), 

m€M{T) Z 

and min |m|(T) such that ^m = yo, (Pn°(l/o)) 

meA1(T) 

where |m|(T) refers to the total variation of the measure m 

|77i|(T) =' sup I J 'ip{x)dm{x) ; ip G C(T) and ||t/’||oo ^ l| • (34) 

Observe that in this framework, the notation |It/’ll00 stands for sup^g^ \ip{t)\. When m 
is of the form m = where G M* and x^ E T (with the x^s pairwise 
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distinct), |m|(T) = so that those problems are natural extensions of {V\{y)) 

and (Po(yo))- This connection is emphasized in [21] by embedding (P”(|/)) and iV^iyo)) 
in the space of Radon measures A^(T), using the fact that 


sup / ii{x)dm{x) ; e C(T), V/c e |0, Pn - 1] \'ip\{khn) < 1 


||a||i if m = (^kdkh„, 

+CXD otherwise. 


We say that mo is identihable through (V^{yo)) if it is the unique solution 
of (V^{yo))- A striking result of [10] is that when $ is the ideal low-pass hlter and 
that the spikes mo = C(o,uXo,u are sufficiently far from one another, the measure 

mo is identihable through V^ijjo)- 

The optimality conditions for and (P^(yo)) are similar to those of the 

abstract Lasso (respectively (15), (16) and (17)). The corresponding dual problems 
are 


resp. 

where 


inf 

pec°° 



2 

2 


sup {yo,p), 




C“"='■ {p 6 L"(T) i iirpiUssi}. 


(vr(v)) 

mm)) 

(35) 


The source condition associated with (V^{yo)) is of particular interest. It amounts to 
the existence of some p G L^(T) such that 


||*h*p||oo ^ 1 and (<l>»(a;o,i.) = sign(ao,!3) for all z/ G {1,..., N}. (36) 

Here, ||<h*p||oo = sup^^T |(<h*p)(t)|. Moreover, if such p exists and satisihes |(<h*p)(t)| < 1 
for all t G T \ {xoq,... ,a;o,Ar}, and <ha;Q has full rank, then mo is the unique solution 
to (V^{yo)) (be. mo is identihable). 

Observe that in this inhnite dimensional setting, the source condition (36) implies 
the optimality of mo for (V^{yo)) but the converse is not true (see [21]). 

Remark 5. A simple but crucial remark made in [10] is that if mo is identihable 
through (V^{yo)) and that suppmo C Qn, then mo is identihable for (Voiyo))- 
Similarly, observe that the source condition for {V^{yo)) implies the source condition 
for the {Voiyo))- 

If we are interested in noise robustness, a stronger assumption is the Non Degenerate 
Souree Condition which relies on the notion of minimal norm certihcate for {V^iyo))- 
When there is a solution to (D[]°(|/o)), the one with minimal norm, determines 
the minimal norm certificate =' When mo is a solution to {V^{yo)), the 

minimal norm certificate can be characterized as 


rj^ = where (37) 

p^ = argmin {||p||2 ; ||<h*p||oo ^ 1, {^*p){xo,u) = sign(ao,,p), 1 ^ u ^ N} . (38) 

peL^iT) 
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As with the discrete Lasso problem, a notion of extended (signed) support ext^oo may 
be dehned and the minimal norm certihcate governs the behavior of the solutions at low 
noise (see [ 21 ] for more details). 

Definition 2. Let mo = '^^=iO:o,udxo,i, identifiable measure for {V^{yo)), and 
f]^ G C(T) its minimal norm certificate. We say that mo satisfies the Non Degenerate 
Source Condition if 

• lho°WI for allt eT \ {xop, • • -Xo.w}, 

• p'^"{xo^y) 0 for allu e 

The Non Degenerate Source Condition might seem difficult to check in practice. 
The following proposition shows that it is in fact easy to check numerically on the 
vanishing derivatives precertihcate. 

Definition 3. Let mo = '^^=iO:o,udxo,^ on identifiable measure for {V^{yo)) such that 
T^o =' rank. We define the vanishing derivatives precertificate as 

=' ^*p^ where 

Py =■ argmin {||p ||2 ; (<l>»(a;o,i.) = sign(ao,i.), (<h»'(a;o,^) = 0, ^N} . (39) 

pGL2(T) 

The following proposition shows that this precertihcate is easily computed by 
solving a linear system in the least square sense. 

Proposition 3 ([21]). Let mo = on identifiable measure for the 

problem (V^{yo)) such that has full rank. 

Then, the vanishing derivatives precertificate can be computed by 

where = T+j* j , (40) 

and = rj,(j(r*^ra;g)“L Moreover, the following conditions are eguivalent: 

(i) mo satisfies the Non Degenerate Source Condition. 

(a) The vanishing derivatives precertificate satisfies: 

• Ihy (^)l < 1 for oil t e T\ {xo,!,. ..Xo,n}, 

• r]y"{xo,u) 0 for all u e {1,..., N}. 

And in that case, rjy is egual to the minimal norm certificate . 

Remark 6 . Using the block inversion formula in (40), it is possible to check that 


Pv 


= $+’*sign(ao,) - n<i>' ($';n<i>' 


<l>l,’*sign(ao,-) 


■ XQ XQ 


(41) 


where fl is the orthogonal projector onto (Im $j;(,)“*-. If we denote by p'fi the vector 
introduced by Fuchs (see (27)), which turns out to be 


p“ = argmin {||p ||2 ; (<F»(a;o,,.) = sign(ao,i/), ^N} , 

pGL2(T) 

we observe that p^ = pf - 
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Remark 7. At this stage, we see that two different minimal norm certificates appear: 
the one for the discrete problem {V^lyo)) which should satisfy (32) on a discrete grid 
Qn, and the one for gridless problem (V^{yo)) which should satisfy (36). One should 
not mingle them. 

3.3. The Lasso on thin grids for fixed A > 0 

As hinted by the notation, Problem {'Pffi{y)) is the limit of Problem {V^{y)) as 
the stepsize of the grid vanishes (he. n —)■ +cxd). Indeed, we may identify each 
vector a G with the measure ma = J2k=o^ (so that ||a||i = |ma|(T)) 

and embed (Vf^{y)) into the space of Radon measures. With this identification, the 
Problem {'P'f{y)) P-converges towards Problem {Vff{y)) (see the definition below), 
and as a result, any accumulation point of the minimizers of {'P'f{y)) is a minimizer 

ot (prfe))- 

Remark 8. The space A1(T) endowed with the weak* topology is a topological vector 
space which does not satisfy the first axiom of countability (he. the existence of a 
countable base of neighborhoods at each point). However, each solution m'f of {'P'f{y)) 
(resp. m’fi of ifPffiiii))) satisfies 

A|m:;|(T) < A|m;|(T) + l||4m; - yf ^ i||sp. (42) 

Hence we may restrict those problems to the set 

X^A^'|meM(T); A|m|(T) ^ i||l/|p| 

which is a metrizable space for the weak* topology. As a result, we shall work with 
the definition of P-convergence in metric spaces, which is more convenient than working 
with the general definition [14, Definition 4.1]). For more details about P-convergence, 
we refer the reader to the monograph [14]. 

Definition 4. We say that the Problem {Vf^iy)) T-converges towards Problem {Vffiij)) 
if, for all m E X, the following conditions hold 

• (Liminf inequality) for any sequence of measures G such that 

supp(m"') C Qn and that weakly* converges towards m, 

liminf ( A|m”|(T) -|- — yp ] ^ A|m|(T) -|- — y\^. 

n—A+oo \ 2, J 2 

• (Limsup inequality) there exists a sequence of measures {m'^)neK G such that 
supp(m'^) C Qn, nfG weakly* converges towards m and 

limsup ( Alm'^l(T) -|- — yp j ^ A|m|(T) -|- — yp. 

n—>-+030 \ ^ / ^ 
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The following proposition shows the T-convergence of the discretized problems 
toward the Beurling Lasso problem. This ensures in particular the convergence of 
the minimizers, which was already proved in [32], 

Proposition 4. The Problem (V^{y)) T-converges towards (V'^{y)), and 

lim inf (V^iy)) = inf {V^{y)). (43) 

n—>-+cx) 

Each sequence {m'^)neK such that is a minimizer of (P”(|/)) has accumulation 
points (for the weak*) topology, and each of these accumulation point is a minimizer 

of (A“(!/))- 

In particular, if the solution mx to (Vff{y)) is unique, the minimizers of (P”(|/)) 
converge towards mx- 


Proof. The liminf inequality of Dehnition (4) is a consequence of the lower semi¬ 
continuity of the total variation and the norm (since <h is weak* to weak continuous, 
— y weakly converges towards <hm — y)\ 

liminf ( Aim’ll(T) + -||4>m” -yp 

n—>-+oo \ 2 

^ A liminfdm^KT)) H— liminf (||<l>m"' — y\\‘^) 

n—>-H-cx) 2 n^+oo ^ 

> A|m|(T) + ^\\^m-yf. 

As for the limsup inequality, we approximate m with the measure = 

hdkh„, where bk = m{[khn, {k + l)h„)). Then, for any tjj e C(T), 


fpdm — / 


Pu-l p 

^ / 

A:=0 [k^ri,ik+l)hn) 
^ uj^{hn)\m\{T), 


{fp{x) — %lj{khn))dm 


where : f i—)■ sup| 2 ,/_ 3 ,|^^ — 'ip{x')\ is the modulus of continuity of ip. Therefore, 

lim„^+oo(m"','^) = {m,ip), and m” weakly* converges towards m. Incidentally, observe 
that Im^KT) ^ |m|(T), so that using the liminf inequality we get lim„^+oo jm^KT) = 
|m|(T). Moreover, by similar majorizations, one may prove that converges strongly 
in L^(T) towards <hm. As a result hm„_^+oo — yp = ||<hm — yp, and the limsup 

inequality is proved. 

Eventually, from (42) we deduce the compactness of X, hence the existence of 
accumulation points, and [14, Theorem 7.8] implies that accumulation points of {mffjneN 
are minimizers of {V^{y)), as well as (43). □ 


The weak* convergence of the minimizers of {Vx{y)) can be described more 
accurately by studying the dual certihcates px and looking at the support of the solutions 
mf to (P”(|/)) (see [21, Section 5.4]). One may prove that m” is generally composed of 
at most one pair of Dirac masses in the neighborhood of each Dirac mass of the solution 

to (^r(l/))- More precisely. 
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Proposition 5. Let A > 0, and assume that there exists a solution to {V’^iy)) which 
is a sum of a finite number of Dirac masses: m'fi = ^ (where a,, 7 ^ D). 

Assume that the corresponding dual certificate rjff = ^*p'fi satisfies \ri’fi'(t)\ < 1 for all 
t eT\ {xi,.. .,xn}- 

Then any sequence of solution o,\,Ah„ to (P”(|/)) satisfies 

limsup (supp(m”)) C {xi,... x^}. 

n^+oo 

If, moreover, m'fi is the unique solution to {Vffiiy)), 

lim (supp(m”)) = {xi,... xat}. (44) 

n—>-+oo 

If, additionally, {yx")”{xif) 7 ^ 0 for some z/ G {1,..., N}, then for all n large enough, 
the restriction of to {x^ — r,Xy + r) (with 0 < r < | \x\^i, — x\y\) is a sum 

of Dirac masses of the form ax,Ay + ax,i+ei,A{i+eiyhr, with ey e {-1,1}, ax,i 7 ^ 0 and 
sign(aA,i) = sign(aA,i.)- Moreover, if ax,i+ei,„ 7 ^ 0^ sign(aA,i+£i,„) = sign(aA,i.). 

We skip the proof as it is very close to the arguments of [21, Section 5.4], Moreover 
the proof of Proposition 13 below for the C-BP is quite similar. 

3 . 4 . Convergence of the extended support 

Now, we focus on the study of low noise regimes. The convergence of the extended 
support for (VqAo)) towards the extended support of (V^{yo)) is analyzed by the 
following proposition. 

From now on, we assume that the source condition for (P^(|/o)) holds, and that 
suppmo C Qn for n large enough (in other words, yo = ‘hg„ao for some Oq G so 

that mo = is a solution of (VoiVo))- Moreover we assume that n is large 

enough so that \xo,u — Xoy\ > ‘^hn for D 7 ^ z/. 

Proposition 6 ([21]). The following result holds: 

lim Vo=v^, (45) 

n^+oo 

in the sense of the uniform convergence (which also holds for the first and second 
derivatives). Moreover, if mo satisfies the Non Degenerate Source Condition, for n 
large enough, there exists G {—1, 0, +1}'^ such that 

ext^”(mo) = supp^(mo) U (supp^(mo) + e'^hn) , (46) 

where supp=^(mo) + e'^hn =' {{xo,u + e(;hn,r]^{xoA) ; 1 ^ z/ ^ A^}. 

That result ensures that on thin grids, there is a low noise regime for which 
the solutions are made of the same spikes as the original measure, plus possibly one 
immediate neighbor of each spike with the same sign. However, it does not predict 
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which neighbors may appear and where (is it at the left or at the right of the original 
spike?). 

The following Theorem rehnes that result by giving a sufficient condition for the 
spikes to appear in pairs {i.e. = ±1 for 1 ^ z/ ^ N). Moreover, it shows that the 

value of e” does not depend on n, and it gives the explicit positions of the added spikes 
for 1 ^ V ^ N . 

Theorem 2. Assume that the operator Ta-g = rank, and that 

mo satisfies the Non-Degenerate Souree Condition. Moreover, assume that all the 
components of the natural shift 

p = (4“ 45* sign(m„(i„)) (47) 

are nonzero, where If is the orthogonal projector onto (Im $ 3 .^)-'-. 

Then, for n large enough, the extended signed support of mo on Qn has the form 

ext=^’^(mo) = {{xu, sign(Q;o,i/))}i^i/<Ar LI + e^hn, sign(ao,i/)}i^i/<Ar (48) 
where e = sign (diag(sign(Q;o))p) • (49) 

In the above theorem, observe that is indeed invertible since has full 

rank. 

Corollary 1. Under the hypotheses of Theorem 2, for n large enough, there exists 
constants > 0, Cn^ > 0 such that for X ^ mini,gi,^Ar |Q;o,ig|; ond for all 
w G L^(T) such that ||tc ||2 ^ Cn^X, the solution to {fP\{y)) is unigue, and reads 
= E!Li(«A,i.dxo,. + l3\,Jxo,,.+ehJ, where 

(^() “ (“0 ) “S" (a() ’ 

where ext„(mo) = U {x^, + e,yhn}i^u^iN, 

e = sign (diag(sign(ao))p), 
sign(aA,7/) = sign(/?A,j.) = sign(ao,j.). 

Proof of Theorem 2. We dehne a good candidate for Pq and using Lemma 1 we prove 
that it is indeed equal to Po when the grid is thin enough. 

To comply with the notations of Section 2, we write 

N P„-l 

^ ^ (Xp^jSxQ „ = ^ ^ O-Ojkdkhni 
u=l k=0 

and we let / =' {i G |0, — 1] ; cio,i 7 ^ 0}. Moreover, for any choice of sign {ei)i^i G 
{-1, +1}^, we set J =' U*e/{b* + £*} and sj = where Sj =' Si+^i =' sign(ao,j) 

for i G /. Since \xo,u — Xoy \ > 2hn for N v, we have Card J = 2 x Card / = 2N. 
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Recalling that A = 0),... 99 (-, (P„ — l)hn) j; we consider the sub matrices 

^/ = (v^(-P^n)) .. .(p{-,Xo^n)^ and Aj\i = {i + ei)hn)^ 

so that up to a reordering of the columns Aj = (^Aj Aj\j^. In order to apply Lemma 1, 

we shall exhibit a choice of (ei)je/ such that Aj has full rank, that v =’ {A*jAj)~^sj 
satishes sign(nj) = —Sj for j G J \ / and ||A}cAjn||oo < 1 • 

The following Taylor expansion holds for Aj\j as n —)■ cxd: 

Aj\i = Ao + hn{Bo + 0(h„)), with Aq = Aj = 

and Bo = Xoy) ... a;o,iv)) diag ((SiJ,... 

= <0 diag((£ij,...(£i^)). 

By Lemma 8 in Appendix, the Gram matrix A*jAj is invertible for n large enough. 


and 


{A*jA, 


.-1 I 

Si 


hr] 


(diag(eii,...,ei^)) V 

-(diag(eii,...,eij^))-V, 


+ 0{l), 


where p is defined in (47), where LI is the orthogonal projector onto (Imand for 
u G |1, A^], iy refers to the index f G / such that ihn = xo^y. Therefore, vj\i has the 
sign of — diag(£i^,.. .eij^)p, and it is sufficient to choose = Si^ x sign(pi,) to ensure 
that sign vj\j = —sj\j for n large enough. 

With that choice of e, it remains to prove that ||A}cAjn||oo < 1- Let us write 


It is equivalent to prove that for k G J'^, |<h*p„(A;h„)| < 1. 


p^ Ajv = Ar 

Using the above Taylor expansion and Lemma 8 in Appendix, we obtain that 
lim pn = Ao’*si - UBo{BoIlBo)~^BoAo'*si 

n^+oo 

= 4:'-sign(a„,) - n4i„(4;,;n4i„)-‘4i;45-sigri(a„,) 

= p^ (by (41)). 

Hence, <h*Pn and its derivatives converge to those of py = and there exists 
r > 0 such that for all n large enough, for all 1 ^ u ^ N, is strictly concave (or 
stricly convex, depending on the sign of p^"{xo^y)) in {xo^y — r,Xo^y + r). Hence, for 
t G {xo^y — r,Xo^y + r)\[xo,y,Xo,y + epy)hn\, we have |<h*p„(t)| < 1. Since by compactness 

max ||? 7 “(t)| ; t G T \ [J{xo^y - r,a;o,,y + r)| < 1 


we also see that for n large enough 

max I |<l>*p„(t)| ; t G T \ |J((ro,j. - r,Xo,y + r) ^ < 1. 


N 


u=l 


As a consequence, for k G J'^, \^*Pn{khn)\ < 1, and from Lemma 1, we obtain that 


^*Pn = Po and [Jy=i{xo^y, xo,y + spy)hn} is the extended support on 


□ 
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To conclude this section, we examine the decay of the constants Cn \ in 
Corollary 1 as n —)■ +cxd. For this we look at the values of ci,...,C 5 given in the 
proof of Theorem 1. 

By Lemma 8 applied to ■l>extn(mo) ^ + 0(hn))f we see that 


e.,„ = ~ )-'<*“ niu, 2 . 

/ hr) 


C2,n I'^/lloo 


(*^ext„ (mo) *^ext„ (mo)) 


-1 I Si 
Si 


C3,n = (ll^i^®it„(mo)IU-2) ^ (^mm \Vk\ 


) 

minfcex \pk\ 




(50) 

(51) 

(52) 


However, the expressions of C 4 and C 5 lead to an overly pessimistic bound on the signal- 
to-noise ratio. Indeed the majorization used in (2.2) is too rough in this framework: it 
does not distinguish between neighborhoods of where the certihcate is close to 1 , 

and the rest of the domain. 

Proposition 7. The constants Cn\Cn'* in Corollary 1 can be chosen as = 0(h„) 
and Cn = 0 ( 1 ), and one has 





(53) 


Proof. The proof of (53) follows from applying (50) and (51) in the expression for ax 
and /3x provided by Corollary 1. Let u =' <F*nt(;, where H is the orthogonal projector 
onto (Im$a;o)-*“ = ker<F*^. In order to ensure (23) we may ensure that : 


(^{jhn) + >^Vo{jhn) \ - A < 0, 


(54) 


for all j G (that is {jhn ^ extn(mo)). 

By the Non-Degenerate Source Condition, there exists r > 0 such that for all 
u e {1,..., iV}, 

^t e {xo,u-r,Xo,u + r), | 77 ^(t)| > 0.95 and |(7“)"WI > ^I( 0 "(a^o,i 3 )|, 

and by compactness sup-jr^ljiv < 1. Since tjq —)■ 77 “ (with uniform 

convergence of all the derivatives), for n large enough, 

Vz/ e {l,...,iV}, Vt e {xo^u-r,Xo^^Pr), |r/(((t)| > 0.9 and |(7/o)"(t)| > ^\{v^y'{xo,^)\, 
(with equality of the signs) and 


sup \r]^\ 

^ k = - 

0 

( sup k“l + i) 

T\U(Li Co,a:o. 12 -t-r) 

L 

V'’^\U(Li Co,!/-»■,aio.ix+r-) / 
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First, for j such that jhn G T + ^"), we see that it is sufficient 

to assume ||*h*|| 00 , 2 !'it’ll 2 < (1 ~ k)\ to obtain (54). 

Now, let z/ G {1,..., iV} and assume that = 1 (so that < 0) 

and that = 1, the other cases being similar. We make the following observation: if 
a function /: (—r, +r) —)■ M satishes f"{t) ^ C for some C < 0 and /(O) = /(h„) = 0, 
then fit) ^ jt{t — hn) < 0 for t G (—r, 0] U [hn, r). 

Notice that u = is a function which vanishes on ext„(mo) (hence at 

and XQ^y + hn)i and that its second derivative is bounded by ||(*h' 0 *lloo, 2 ||ia|| 2 - Moreover, 
= Voixo^u + hn) = I and sup(,,^^ \{v^y\xo,u) < 0. Thus, for 
2II($'M*T’‘'^^ , we may apply the observation to a;(- — Xq^i,) + X{Pq{- — Xq^i,) — 1 ) so 


< 


Iblh _ 

A ^ 2||($'') 

as to get 


u{t) + Kvoit) - 1 ) ^ (^||(*h")*||oo, 21^12 + A^(i 7 “)"(a;o,,.)^ (t - xo^^){t - xq^^ - < 0 

for t G {xo^y - r, Xo,u] U [xq^u + /i„, Xq^^ + r). 

On the other hand, the inequality —oj{t) — A(? 7 g (t) +1) < 0 holds for 14 ** 1100 , 2 ||'ii ’||2 < 
1.9A. As a result (54) holds for all j such that G (xo^y — r, Xo^y + r), provided that the 
signal-to-noise ratio satishes ^ c, where c > 0 is a constant which only depends on 
min^|(77~)"(a;o,^)|, || 4 >*||oo, 2 , ||( 4 >")*||oo ,2 and \v^\. In other words, 

including the condition involving C 3 ^„, we may choose Cn'^ = min(c 3 ^„,c) = 0 ( 1 ). □ 


4. Abstract analysis of the Lasso with cone constraint 

This section studies a simple variant of the Lasso with cone constraint in an 
abstract setting. The results stated here shall be useful in Section 5, since this variant 
turns out to be the Continuous Basis-Pursuit when the degradation operator is a 
convolution with an impulse response and its derivative. Similarly to Section 2, we 
consider in this section observations in an arbitrary Hilbert space Ti. 

4 . 1 . Notations 

Given a parameter h > 0, we consider the cone generated by the vectors (1, |) and 

C/i =' I (c, d) G M X R ; c ^ 0 and — c— + \d\ ^ 0 1 . (55) 

We also dehne the cone C[ as the set of vectors (a, b) G R^ x R-^ such that for all 
/c G |0, P -1] {ak,bk) eCh. 

Now, given a vector (ao,bo) G Cf" (i.e. 'ik G |0, P — 1|, ao,A: ^ f l^o,fc|), we observe 
Uq = Aao + P&o, where A : R^ —)■ Ti and B : R-^ Ti are linear operators, or its noisy 
version y = yo + w where w & Ti. To recover (oq, &o) from y or |/o, we consider the 
following reconstruction problems: 

1 

o\y - Ala - Bb\l + A||a||i, 

{a,b)&CP 2 


iQxiy)) 
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min ||a||i such that Aa + Bb = yo- (Qo(l/o)) 

(a,6)eCf 


Our main focus is on the support recovery properties of {Qx{y)). Precisely, we split 
the “support” of (a, b) G C[ into several parts: 


I =' supp(a) =' {i G |0, P — 1] ; Oj > 0} 

= u 

where =' G / 


; a* + ^bi > o| , G / ; - ^6* > 0 


(56) 

(57) 

(58) 


Observe that in general O ^ 0. If {a\,bx) is a solution of {Qx{y)), we 
say that we have exact support recovery provided that I^^\ax,bx) = I^^\ao,bo) and 
I^^\ax,bx) = I^^\ao,bo). 

Remark 9. The notation which might seem a bit obscure at this point, shall 

become clearer in the next section. It turns out that when considering the Continuous 
Basis-Pursuit on a grid with stepsize h > 0, points i in correspond to Dirac masses 
which “tend to be on the right”, that is they do not coincide with the left half-grid point 
ih — Similarly, points in correspond to Dirac masses which “tend to be on the 
left”, as they do not coincide with the right half-grid point ih + ^. In fact, if i G 
it correponds to a Dirac mass at the right half-grid point: and if i G \ 

it correponds to a Dirac mass at the left half-grid point: S^h_h. If i G fl it 
correponds to a Dirac mass which may belong “freely” to the interval {ih — -|- |). 


4-2. Parametrization as a positive Lasso 

To characterize the solutions of (Qa(|/)) and (Qo(l/o)); h is convenient to 
reparametrize the problem as a Lasso with positivity constraint. Indeed, let us write 
for all i E |0, P — 1], 



I fai + ffeA 


In the following, we dehne the linear map 


Hh: 



(59) 


(60) 


It is clear that 
(a,6)GCf, 


(oj, bi) G Ch if and only if r* ^ 0 and h ^ 0. 


Moreover, given 


r = {*G[0,P-l]; (r„h) = (0,0)}, jW = {, e [0, P - 1] ; r, > 0} , 

and = {i G |0, P - 1] ; /* > 0} . 
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Therefore, Problems {Qx{y)) and (Qo(l/o)) are respectively equivalent to the Lasso 
and Basis Pursuit with positivity constraint: 


min A 

(r,l)&{R+)Px(R+)P 




(Qxiy)) 


and min 

{r,l)G{R+)Px(R+)P 



such that 



yo, 


{Qo{yo)) 


where A = (^ + A- ^ 7 ^, 

Observe that there is “support recovery” of (ao,&o) through (Qx(y)) if and only if 
there is support recovery of (ro, lo) through (Qx(y)). But precisely, as we shall explain 
below, the characterization of minimizers and the support recovery properties of the 
Lasso with positivity constraint (Q\(y)) are quite similar to those exposed in Section 2. 

The regularization term may be written as J : ^ M U {+cxd}, where for all 

(r, /) e M-P X M-P, 

n LI- I + h) if rA 0 and /A 0 for all i G |0, P - 1], 

’ I +CXD otherwise. 

p-i p-i 

2 = 0 2=0 


with 


j{x) = sup {qx ; g ^ 1 } 


X if a; ^ 0, 
+CXD otherwise. 


Hence, the subdifferential of J is the product of the subdifferentials djir'i) and dj{li) 
for 1 ^ i ^ P — 1, where 


di{x) 


{ 1 } if a; > 0 , 
(— 00 , 1 ] if a; = 0 . 


That is quite similar to the subdifferential of | • | at a; G M which is —1, [—1,1] or 1 if 
a;<0,a; = 0ora;>0 respectively, and one may adapt all the results of Section 2 to 
the Lasso with positivity constraint. It essentially amounts to replacing the conditions 
It^Ioo ^ 1 with max ?7 ^ 1 (and similarly for strict inequalities) wherever they appear. 
We leave the detail to the reader, and in the following, we use those results freely to 
derive the properties of the Lasso with cone constraint {Q\{y)). 


4-3. Optimality conditions 

Applying the results (or their straightforward adaptations) of Section 2 to {Qx{y)) 
and (Qo(yo)), then composing by we immediately get the following results. 
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Proposition 8. Let y eTL, {a\, b\) G C[, and I = I {ax, bx) ■ Then {ax, bx) is a solution 
to {Qx{y)) if and only if there exists qx & TL such that 


h 


max {A* + -B*)qx ^ 1, and max {A* - -B*)qx ^ 1, 


h 


(61) 

/ V ^ / 

(A*(r) + —Bji^r))qx = l/(r), and {A*^^ — —B*(i))qx = 1 /( 1 ), (62) 


A 


A* 

B* 


qx + 


A* 

B* 


{Aax + Bbx -y) = 0. 


(63) 


Similarly, (ao,&o) ^ is a solution to {Qo{yo)) if and only ifAao + Bbo = yo and there 
exists q eTL such that 


^(A* + ^ 1, and max ^(A* - ^B*)q'^ ^ 1, 

(A*(r) + —B*^„))q = l/(r), and (Aj{i) — —B*(^i))q ^ 1, 


where I = /(oq, bo). 

The corresponding dual problems are given by 


inf 

q&D 



2 


sup {y,q) 

qeD 


where D = 


< g G "H ; max 
( fce[o,Pn-iI 


{A*q)k + —\{B*q)k 





(64) 

(65) 


{So{y)) 

( 66 ) 


Again, if the inequalities outside the support are strict, it is possible to ensure the 
uniqueness of the solution. 

Proposition 9. Under the hypotheses of Proposition 8, if ^(A + |i?)/{r) {A — i^B)pi)^ 
has full rank and if qx (resp. q) satisfies 


Vie/', (A’qOt + ^\{B’ilx)k\<l, 

(67) 

Vi \((V + < 1, 

(68) 

Vi £/W \/<■>, ((V-|b-),,),<!. 

(69) 


then {ax,bx) (resp. (oq, &o)j is the unique solution to {Qx{y)) (resp. {Qo{yo))J- 


4.4- Low noise behavior of C-BP 

The Theorem of Fuchs [26] for the Lasso (see Remark 3) extends to this setting as 
follows. 
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Proposition 10. Let (oq, &o) ^ ^/T \ {0} 


has full rank, and let 


A = (71 + (A - 15),(.) 


T =■ min i 6o,i + ; * e oo) A i 6o,i - ; i e oo) 


h 


h 


(70) 


T/ien ^/iere exists constants > 0 such that for A ^ ||w|| < 

the solution to {Qx{yo + w)) is unique, satisfies I^^\b\,ax) = I^^\bo,ao), I^^\bx,ax) = 
Oo); it reads: 


ax 

bx 


Oq 

bo 


+ HhAlw - AA(AA) ^-s, 


where Hh is defined in (60). 

In general, the conditions of Proposition 10 do not hold, and the support at low 
noise is strictly larger than {I^^\ao,bo), I^^\ao,bo)). This support is governed by the 
minimal norm certihcate. 

Definition 5 (Minimal norm certificate). Let (oq, bo) G Cf). Its minimal norm certificate 

j^* _j_ 

^ ' go where go is the solution to {So{y)) with minimal norm. The 


y. def. 

IS Co = 


71* - §5 


extended support is ext*^''^^(ao, &o) = (ext*^''^(ao, 6o), ext^^^(ao, &o)) ? where 


extW(ao, bo) = <1 j e [0, P - 1] ; ((A + ‘-B*)qo), = A , 


h 


ext(^)(ao, bo) = <{ j G |0, P - 1] ; ((A - '■^B*)qo)j = 1 


h 


(71) 

(72) 


From the optimality conditions, if (ao,&o) is a solution of (Qo(l/o)) then C 
ext^'’i(ao, &o) and C ext‘^*i(ao, &o) (where / = I{ao,bo)), and go can be characterized 
as 

^A + |P*' 

h 
2" 


go = argmin <( ||g ||2 ; 

q&'H 


? e dJ{ro,lo) 


(73) 


Lemma 2. Let j(i) (y |o^ p — ij^ and (ao,&o) G C^. Assume that (Jh), =■ 

(A)(ao,6o),/^')(ao,&o)) is such that 

j(r) ^ j(r)^ j(l) ^ j(l) dj. ^ j 

has full rank. Define 


Uj(r) \ d^f. ig where s =' 

VjO) 


Then 




J^-’i) is the extended support of (ao,&o) if ond only if the following two conditions 

hold: 
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< 1 . 


for all j G \ Uj ^ 0, and for all j G \ Vj ^0. 

(A + J fuj(r) 

SA-lB)*,rJ 

Moreover, in that ease, the minimal norm certificate is given by 

Co = -AH*{AlAh)-^s. 

The proof is identical to the one of Lemma 1, therefore we omit it. We are now in 
position to describe the behavior of (Q\(y)) at low noise in the generic case: 

Theorem 3. Let (ao,&o) ^ (®+)^ \ {0} an identifiable signal, {J^^\ = 

ext(''^)(ao, &o) such that Ah =' + |i?)j(r) — has full rank. Let 

/1\ 

G and assume that for all 


Uj(r) \ <w. ig where s =' 


Vjm 


\L 


j G \ Uj > 0, and that for all j G \ > 0. 

Then, there exists constants OB') > 0, > 0 such that for 

A ^ min |boj + ; i G (oq, 6o) 1 U | bo,i - \aoj ; i G (oq, 6o) \ (74) 


h 


h 


and ||t (;||2 ^ the solution {a\,bx) to {Qx{y)) is unique, I^^\ax,bx) = 

I^^\ax,hx) = and it reads 


ax 

bx 


ao 

bo 


+ HhAfw - \HhiA*hAh) ^s, 


where Hh is defined in (60). 


5. Continuous-Basis Pursuit on thin grids 


Facing the same inverse problem as in Section 3, but this time assuming that each 
{1 ^ u ^ N) is positive, we aim at recovering mo using the Continuous Basis- 
Pursuit proposed in [23]. Given a grid Qn as in Section 3, the goal is to reconstruct 
a measure m = ai5ih„+ti where U G which estimates mo- Applying a 

Taylor expansion and setting 6* = tjOj, the authors of [23] are led to solve 

min h\y - - ^g^bf + \\\a\\i {Qliy)) 

min ||a||i such that + ^'gj = yo- (Qo(l/o)) 

which are particular instances of (Qx(y)) and (Qo(l/o))- The dual problems are 
respectively: 


inf 

g€D" 


y 


-Q 


sup {yo, q) 

qeD^ 


2 


(£"(</)) 

(£S(vo)) 
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where D” =' G L^(T) ; max {^*q){khn) + q)'{khn)\ ^ ll , (75) 

( fcG|[0, P„-l] 2 J 

To study the behavior of the solutions to these problems as n increases, we aim at 

applying the results of the previous section, and in particular Lemma 2, in the setting 

where H = L\T) and (A, B) = 


5.1. The positive Beurling Lasso 


The situation with the continuous basis pursuit on thin grids is quite similar to 
the situation of the Lasso. Still, the non-negativity constraint on the components 
{ak)o<^k!iPn passes to the limit, and the appropriate limit model is the positive Beurling 
Lasso, 

+ Am(T), (Qr(l/)) 

m£M+{T) I 

and min m(T) snch that <hm = yo, (Q(f(|/o)) 

mG>l+(T) 

where A^^(T) refers to the space of positive Radon measures. The indicator fnnction 
of positive measures plus the total mass may be encoded in the qnantity: 


m(T) -|- <.Ai+(T)(^^) = sup Iy 'ip(t)dm(t) ; ip G C(T) and m.&^ip{t) ^ l|. (76) 

As a result, the characterization of optimality, the notions of minimal norm certihcates 
and extended support may be adapted from Section 3.2 in a straightforward manner, 
replacing condition H^^Hoo ^ 1 by snp^g-j-/i(t) ^ 1 where p = for q G L^(T). For 
instance, up to the addition of a constant, the dnal problems to {Qpp{y)) and (^“(i/o)) 
are respectively: 


inf 

q€D°° 



sup {yo, q) 

q&D°° 


isrw) 

(£S°M) 


where 


B>°°={qeL 


■(T) ; max(t'q)(t) s: 


(77) 


5.S. The limit problem for thin grids 


To consider the limit of (Q”(|/)), let us recall that we obtain a measnre from the 
vector (a, b) G by setting 


Pn-l 

m= ai6ih„+bi/ai (78) 

i=0 

with the convention that fej/a* = 0 if a* = 0. It should be noticed that bi/oi E 

We rely again on the notion on T-convergence to express the convergence 
of {Qx{y)) towards (Q^(|/)). As before, we may restrict the problems to =' 
{m G Ad + (T) ; A|m|(T) ^ ||||/P} which is metrizable for the weak* topology. 
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Definition 6. We say that the Problem (Q^iy)) T-converges towards Problem (Q“(|/)) 
if, for all m G the following conditions hold 

• (Liminf ineguality) for any seguence of measures (m"')neN £ of the form (78) 

with G such that m” weakly* converges towards m, 

liminf rA||a(”)||i + - |/||^) > Am(T) + ^||$m - yf. 

n—>+oo \ Z =7,L j 2 

• (Limsup ineguality) there exists a seguence of measures {m'^)n£n G of the 

form (78) with G Cj^f such that m" weakly* converges towards m and 

limsup rA||a(”)||i + - 1/11^1 ^ Am(T) + h\<^m-yf. 

n^+oo \ ^ 


Proposition 11. The Problem {Qf{y)) T-converges towards {Q)f{y)), and 

lim inf {Ql{y)) = inf (Q“(|/)). (79) 

n—>-+oo 

Each seguence {rn^fjnen such that is a minimizer of (Qf^y)) has accumulation 
points (for the weak*) topology, and each of these accumulation points is a minimizer 

of (srfe)). 

In particular, if the solution to {Qf^iy)) is unique, the whole sequence (m)()„eN 
converges towards 


Proof. The proof is the same as for Proposition 4 with minor adaptations, observing 
that ||u*'’^^||i = mn{T). For the liminf inequality, let {m'^)neN be of the form (78) which 
weakly* converges towards m. We notice that and 


Pn-l 




i=0 


h 


'Pn-l 


(T) < ^ I E T’ < 


h„. (1 


i=0 


2\\2 


+ 1 —)■ 0, 


so that (strongly) converges towards 0 in L^(T). Moreover, = 

and for all tf G C(T), 


IPn-l 


Pn-l 


^ ^ ^ihn+bifai ^ ^ Cbi6ih„,l) 


i=0 


i=0 


Pn-l 


a^f'\'il){ihn + hi/ai) - ip{ihn)) 


i=0 

Pn-l 




i=0 


hr 


where : t ^ sup| 2 ,/_ 2,|^4 |t/’(a^) — 'f’ix')] is the modulus of continuity of ip. As a result, 

^ 0 and ^ihn weakly* converges to m. Hence, 

weakly converges towards <Fm in L^(T). To sum up, — y weakly 

converges towards <Fm — y and we conclude as before. 

For the limsup inequality, the only difference is in the construction of for the 

limsup inequality: it is sufficient to choose = m{[khn, {k + l)hn)) and = 0 for 

all A; G |0, - 1], □ 
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Though Proposition 11 states the convergence of the solutions of (Q”(|/)) towards 
those of (Q“(|/)), it does not describe the supports of the solutions. We now study 
the convergence of those supports using dual certihcates and the optimality conditions 
(Proposition 8). In this context, a dnal certihcate is determined by a fnnction 
fi = e C(T) where q G T^(T), and 


= I* G [0, Pn-lj;h> -^a,| C I* G [0, P„ 
= I* G [0, P,, - 1] ; 6, < C I* G [0, P„ 


11 ; (t + (ihn) 

11 ; (t- (ihn) 



To snm np, we shall exploit the following observations 

• if (/i + ^/i') (ihn) = 1 bnt (/i — (ihn) < 1, a spike may appear at ihn + 

• if (jU — {ihn) = 1 bnt (/i + ^/i') {ihn) < 1, a spike may appear at ihn — 

• if (/i + {ihn) = 1 and (/r — {ihn) = 1, a spike may appear anywhere in 

the interval [ihn — ihn + ^]- 

The following lemma is central in onr analysis. We consider a seqnence of fnnctions 
{fi"')nm and for 0 < r < \xu — Xn'\, G {1,... ,N}, we stndy: 

G n {Xn -r,Xn + r) ; + yh”'^ (f) = l| , 

=' G n {x^ -r,Xn + r) ; - yh”'^ W = l| • 


Lemma 3. Let {xi,... ,xn) G pairwise distinet, and let {fi^}nefi G (C^(T))^ be a 
sequenee of funetions which converges uniformly towards some /i°° (and similarly for the 
derivatives) such that for allv & {1,..., N}, p°^{xif) = 1 and for all t G T\{a;i ,... ,xn}, 
p,^{t) < 1. 


(i) Then 


limsnp <j t G ; T"'{t) + = 1 1“ P • • -^xn}- 

n—¥+oo 


(80) 


In particular there exists no G N such that for n ^ Uq 
hr. " ^ 


N 


teGn] /i”W + ylh”'(f)l = 1 f = U U G [j{Xn-r,Xn + r). 


u=l 


u=l 


Assume moreover that for all n E N and all t G Gn, hG{t) + ^\hG'{t)\ ^ 1- For each 

z/G{1,...,7V},. 

(a) If {fi°°)” {xn) 7 ^ 0, then there exists Uq G N such that for n ^ no, each set Snl{r) and 
Sn^u{r) is of the form 0, {ihn}, or [ihn, {i + l)h„}, and if both sets are nonempty: 


maxPW(r) ^ minP^^ (r). 
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(Hi) If 7 ^ 0, then there exists no G N such that for each n ^ no, Sn}u{r) = 0 

or S'n,i(r) = 0. 

(iv) If {pi°°)‘^'^\xy) 7 ^ 0, the set o/n G N such that Sn}u{r) = {{in — ^)hn,inhn} and 
Sn^u{r) = [inhm (*n + l)^n} (with thc samc in G |0, Pn — l]y) is finite. 


Proof. Observe that both pp + ^pp' and pp — ^pp' converge nniformly towards pp° as 
n —)■ +CXD (and similarly for the derivatives). 


(i) For all r G (0,r), by compactness, sup{/i°°(t) ; t G T \ lj(a;,y — r, + r)} < 1. 

Thus by uniform convergence there exists no G N such that for all n ^ no, 
{pp ± < 1 on T \ — r, + r), and the hrst claim is proved. 

(ii) If moreover {pp°y'{xv) 7 ^ 0, it is in fact negative. Choosing r G (0,r) small enough 

and then n large enough, we may assume that pP” < —k^ in {x^ — r, x^+r), for some 
ko > 0, and by (80) that 5'i^^(r)US'i';l(r) C {xiy — r,Xiy+r). By uniform convergence, 
^n/> _|_ ZkL|^«(3)| fQj; ^ large enough, so that both functions pP + ^pp' and 

pp — ^pP' are strictly concave in {x^ — r^x^ + r). This implies that Sn]^{r) (resp. 
5'n,i(r)) is of the form 0 , {ihn}, or [ihn, {i + I)hn}. 

Observe also that pP + ^pp' — {pP — ^pp') = hnpP'■ Since the function pp' is 
strictly decreasing in {x^ — f,Xn + r), it vanishes at most once. If Sn]^{r) 7 ^ 0 and 
>S'n,i(r) 7 ^ 0, it must change sign in {x„ — f^x^ + r) and thus it vanishes exactly 
once, at some f G {x^, — f,Xy + f). Then for t G {Xy — r,f,), 

(;■" - YA>”')(i) = + yA-^Xi) - fcn(A‘”)'(A) < 1 - fc..(A<")'(i) < 1 

SO that minS'i*l(r) ^ Similarly maxS'ny(r) ^ 

(hi) By contradiction, assume that the set of n' G N such that Slf,\{r) 7 ^ 0 and 
7 ^ 0 is inhnite. We may extract a subsequence n = n'{m) such that there 
exists in,jn G |0, Pn — 1] (denoted hereafter i,j) with ihn G Sn}v{rm), jhn G S'i'l. 
Combining the Taylor expansions of p” and {p,'")' around ihn (resp. jhn), we get 


1 ^ ^''{{l + l)hn) - %/i"'((* + l)hn) 


— p"'{ihn) + hnp'' {ihn){^ — —) phn^p^ {ihn) ~ 2 ^ phn^p''^^\ihn)oi3 


P hn^ I p^^^^ihn P thn) 

1 > - l)fc„) + yA*”'((i - l)ft„) 


(!-()=> (!-«)- 


3! 


2! X 2 


= p'^{jhn) - hn/i” {jhn){l - “) Phn /i” {jhn) 


=1 


+ h/ / P^^'^^jhn-thn) 



dt, and 


-hjp''^^\jhn)as 
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where is defined in (96). Now, let n —)■ +cxd. By (80), ih^ —)■ x^, and jhn —)■ Xy, 
and using the uniform convergence of towards dividing by hn^, 

we obtain respectively 0 ^ x ^ and 0 ^ {fi°°Y^\x^) x thus 

(/i°°)(3)(a;,) = 0. 

(iv) Assume by contradiction, that the mentioned set is infinite. For such n, a Taylor 
expansion at ihn yields (we write i for in). 

1 =,!”((; +i)fc„) - + i)M 

= + YA<"'(a„) +73fc„y”‘”(»A„) + 74A,.V<‘‘’(ifc„) 

= 1 

+ hn^ pG^^\lhn + thn) ^ud 

1 = pG{{l - l)hn) + yp"'((* - l)hn) 

= p^{lhn) - y/i"'(*h,7) -73/^n'/i"® (*h„) + l,hn%^^^\lhn) 

'' - J- -' 

+hj + tK) 

with 7fc = ^ — (fc-i)!x 2 - Summing both equalities, dividing by and taking the 
limit n —)■ +cxd yields {p.°°)^'^\xn) = 0. 

□ 


This other lemma focusses on the limit of the sets D'^ defined in (75). 

Lemma 4. As n —)■ +cxd, the sets D" converge towards defined in (77) (in the 
sense of set convergence). 

Proof. We observe that i?” C 77” C F", where 

E" |, € L^(T) ; max(r,)(«) + i, 1 j , 

F” = I g e L2(T) ; max ^*q(khn) ^ 1 

( fcG|[0,Pn-l] 

SO that it suffices to prove that F” and F” converge towards D°°. On the one hand, it is 
clear that F°° = sequence F” is non-increasing. On the other hand, 

it is possible to check that F°° = IJneN-^”’ sequence F” is non-decreasing. As 

a consequence, the claimed set convergences hold. □ 
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Let us recall that the dual problem to (Q”(|/)) is the projection onto the closed 
convex set 

D" " I? e ; (4-,)(*/!„) + L|(r,)'(*ft„)| ^ l|. 

Since the set convergence of (see Lemma 4) implies the convergence of the 
projections onto D'^ (see [29], or [21] for a direct proof in a similar context), we obtain: 

Proposition 12. Let qf (resp. q'^) be a solution of (resp. {Sff{y))), and 

pjf = (resp. /i“ = Then 


lim g" = q^ for the L^(T) strong topology, 

n^+OD 


lim 

n—>-+C 50 


oo(k) 

Tx 


in the sense of the uniform convergence, for all k eN. 


The following proposition states that in the generic case, one may observe up to two 
pairs of spikes for each spike of the solution of the positive Beurling-lasso. As before, r 
is chosen such that 0 < r < | min,^^,^/ \x,y — x,yi\. 

Proposition 13. Let A > 0, and assume that there exists a solution to (Qf^(y)) which 
is a sum of a finite number of (positive) Dirac masses: where Oj, > 0. 

Assume that satisfies |/i“(t)| < 1 for allt eT \ {xi,... ,xn}- 

Then any sequence of solution (i\,Ah„+b),^i/ax,i to {Qf{y)) satisfies 


lim sup (supp m”) C {xi,... xjq}. 

n—>-+00 

If, moreover, is the unique solution to {Q)f{y)), 


lim (supp(mj()) = {xi,.. .xat}. (81) 

n—>-+00 

If, additionally, {ijl^)'\xif) ^ 0 for some z/ G {1,..., N}, then for all n large enough, 
the restriction of to {x,, — r,Xu A- r) is a sum of Dirac masses whose configuration 
is given in Table 1, and if {plx‘)^^\xu) f 0, then only the cases indicated with (*) may 
appear. 


Proof. By Proposition 12, we know that the dual certihcates pdf converge towards pif. 
By Lemma 3 and the optimality conditions, we have thus lim sup„^_,_oo (supp(m”)) C 
{xi,... ,X]xi}. If mf is the unique solution, assume by contradiction that 
hminf(supp(m”)) C {xi,..., xat}. Then there is some z/, some e > 0 such that (up 
to a subsequence) (supp(m”)) fl {x^, — e,x,y + e) = 0. This contradicts the T-convergence 
result (Prop. 11) which ensures that mf converges towards for the weak* topology. 
As a result hm„_j.+oo(supp(m”)) = {xi,. .. ,xn}- 

If {pTY'i Xy) 7 ^ 0, Lemma 3 ensures that the sets Sii,l{r) and S'i';l(r) are of the 
form 0, {ihn}, or {ihn, {i + l)hn}. Moreover, since hm„^+oo(suppm”) = {xi,... ,xn} 
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Number 

of Dirac 

masses 

Saturations of the 
certificates 

{sYl/sYl) 

Possible Dirac Locations 

One 

{ihn}/% or 0 /{i/r„} 

ih„ + £n^, with en & {—1,1} (*) 

{ihn}/{ihn} 

ihn + U, with < n < 


{(i - l)hn,ihn}/% 
or 

(li/{ihn, (i + 

(](i - e„)h„ + with £„ G {-1,1} (*) 

Two 

{ihn}/{jhn} 

(ihn + - %!-]), i < j 


{ih„}/{jhn, (i + l)hn} 

or 

{(i - l)hn,ihn}/{ihn} 

({i — £^n)hn £n , ihn + ti'j , Sn G { — 1,1}; — ^ li ^ 


{ih„}/{jhn, (j + 

(ihn + k^,jhn - ^,{j + l)hn - , with i < j 

Three 

{(i - l)hn,ihn}/{jh„} 

({i - l)hn + t^,ihn + tl^,jhn - ■^]), with i < j 


{(i — l)hn,ihn}/{ih„, {i + l)hn} 

(]{i — l)hn + ; ihn + ti; (i + l)h'ri — ^ 

Four 

{{i-l)hn,ihn}/{jhn, (j + l)hn} 

(^{i - l)hn + tl^,ihn + tl^,jh„ - tlf, {j + l)hn - i < j 


Table 1: Number of Dirac masses that may appear if 7 ^ 0. For the sake of 

the simplicity of the table, and since we focus on the saturations of dual certihcates, we 
regard sums like as “two” Dirac masses. 

we must have Sn,l{r) 7 ^ 0 or 7 ^ 0. Using the fact that maxSnl{r) ^ min5'n,i(r), 

one may check that the only possible saturation points of /i” + and /i” — 
given in Table 1. The optimality conditions of Proposition 8 imply that is at most 
a sum of Dirac masses at those locations. 

If 7 ^ 0 the third point of Lemma 3 implies that for n large enough, 

Snl{r) = 0 or Sn}u{r) = 0 (but not both). Hence there are at most two (successive) 
saturations, produced either by p^ + ^Px' or by Px — ^p'x- ^ 

Remark 10. Proposition 13 states that the support of the C-BP on thin grids actually 
depends on the properties of the dual certihcate /i^ of the (positive) Beurling Lasso. 
The condition {p'^)"{x„) 7 ^ 0 seems to be overwhelming, if not generic, and it is ensured 
for instance if A is small and the Non-Degenerate Source Condition holds (see [21]). As 
for the condition {p'^Y^Y^i') 7 ^ 0 ) if seems to be generic, as there is nothing to 
impose = 0 in the positive Beurling Lasso. As a result, in practice, one 

does not observe all the eonfigurations given in Table 1, and only the eases indieated 
with (*) appear, the case of two spikes being again overwhelming. 

This means that when approximating the positive Beurling Lasso with the 
Continuous Basis-Pur suit, one generally sees two spikes instead of one, and those spikes 
are at successive half-grid points: {ih + ^,{i + l)h + |) or {ih — (f -|- l)h — |). 
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Now, we focus on the behavior of the Continuous Basis Pursuit at low noise. As 
for the Lasso, this analysis is more difficult in whole generality, since it involves the 
minimal norm solutions of nonlinear problems, in which it is difficult to pass to the 
limit. Therefore, we are led to assume that {xop, ..., a;o,Ar} C Qn, and the measure now 
reads ruo = 

The following property ensures that mo is a solution to (Qo(|/o)) for each n large 
enough. 

Lemma 5. Assume that there exists a function p, G Im<h*, such that for all t G 

T \ {xo,!,..., Xo,n}, h(f) < 1 and 

Vz/ G {1,..., N}, p{xo^„) = 1, p'\xo^u) 7^ 0, = 0, p^‘^\xo^u) > 0. (82) 

Then, for alln large enough, p is a dual certificate for mo = EILi c^o^udxo,,, far (Qo(l/o)); 
and mo is a solution to (Qo(yo))- Moreover, ifT^Q has full rank, this solution is unique. 

Remark 11. The condition p^^\xo^u) = 0 is natural since our aim is to build a certihcate 
which is valid for all n, hence Lemma 3 applies with Sn]y{r) ^ 0 and S'n,l(r) ^ 0. 

Proof. Let u G {1,...,A^} and G (0,r) such that p''(t) < 0, and p^‘^\t) > 0 in 
{xo,u — ru,Xo^u + r,f). We shall prove that p{khn) + ^\p'{khn)\ < 1 for all k such that 
khn G {xq^i, — r^, Xo^v + r,f) \ {xo^u}- To simplify the notation, we assume without loss of 
generality that xq^^, = 0 and we write r = r^. The variations of p and its derivatives are 
given by the table below: 


t 

—r 0 r 


+ 


--0 — ^ 

p” 

p''{—f) < 0 < 0 

h"(o) 

p' 

-■-0- -— . 

p 

1 


Let us observe that the function 9 : t pit) — ^p'it) is (strictly) decreasing in [0,r), 
since 

Vt G (0, f), 9'it) = ]- {p'it) - tp”{t)) = ]- f {p"{u) - p"{t)) du < 0. (83) 

^ ^ Jo '' ---' 

<0 
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Hence, for all k such that khn € (0,r), 

/i(/ch„) - ^p!{khn) = p,{khn) - ^^p!{khn) + ^^-^^p!{khn) < 1. 


=6»(fc/i„)<6»(0)=l <0 

On the other hand, 9 is (strictly) increasing on (—f, 0] since 

Vt e (-r, 0), e'{t) = ]- (/i'(t) - tp!'{t)) = \ [ du > 0. 

^ ^ Jo '' -V-' 

<0 

As a consequence, for all k such that khn G (—h, 0), 

p,{khn) + ^p!{khn) = p,{khn) - ^^p'{khn) + ^—^^^fi'{khn) < 1. 

'-'-V-' 

=e{kh„)<e{o)=i ^0 


(84) 


(85) 


( 86 ) 


Thus we see that fr{khn) + ^\ir'{khn)\ < 1 for all khn G (—r,r) \ {0}, and we proceed 
similarly on all the intervals of the form {xo,u—ru, xo^n+r^)- By a compactness argument, 
there exists a constant /3 < 1 such that pit) ^ /9 for all t G T \ xo,u + ru)- 

For n large enough, the inequality ^ (sup^g-j- \p'it)\) <1-/3 holds, and we see that 
p{khn) + ^\p'{khn)\ < 1 for alH G T \ {jn=i{^o,u - ru,Xo^n + r^). 

As a conclusion, we see that p is a valid certihcate for (oq, 0) (see the optimality 
conditions of Proposition 8), thus (ao,0) is a solution of {Q^iyo))- □ 


Now, we consider the limit of the minimal norm solutions of iS^iyo))- In general, 
they do not converge towards the minimal norm solution of (^^(("(i/o)), and we are led to 
introduce a new variational problem to carry the study further. 

Definition 7 (Third derivative precertihcate). Given tuq G Ad(T), we define the third 
derivative preeertificate as px = ^*qT where 

qr =' argmin | ||g|| 2 ; Vf G {1,..., N}, i^*q)ixi) = 1, 

gGL2(T) L 

(<F*g)'(a;i) = 0 and i^*q)^^\xi) = ol, (87) 


whenever the above set is not empty. 


It is clear that the set dehned in (87) is a closed convex set. It is nonempty for 
instance if the conditions of Lemma 5 hold. Note that qx corresponds to a quadratic 
minimization under linear constraint, and can hence be computed by solving a linear 
system. 


qx 



V / 

V 0 / 





- fltlf (4lffM><f )-‘4<f p;?, 


( 88 ) 

(89) 
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where H is the orthogonal projector onto (Imra.g)-'-, and Fa-g = 

Definition 8 (Twice Non-Degenerate Source Condition). We say that tuq satisfies the 
Twiee Non-Degenerate Souree Condition (TNDSC) if qr in (87) is well defined and if 
it satisfies, for pt = ^*qT, 


Vt G T \ {xo,!,..., a;o,7v}, hr(^) < 1, 
Vz/e {1,..., iV}, p'f{xo^u)<0 and /i^^(a;o,y) > 0. 


Observe that if the Twice Non-Degenerate Source Condition holds, the hypotheses 
of Lemma 5 are satished and tuq is a solution to (Qo(|/o)) for n large enough. In fact 
the associated minimal norm certihcates (which thus exist) converge towards pt- 

Proposition 14. Let mo G A1(T) satisfy the Twice Non-Degenerate Souree Condition 
(and pt the corresponding Third derivative (pre)certificate). Let he the minimal norm 
solution of {So{yo)), and pfi = ^*qo. Then, 


lim gg = qT for the L^(T) strong topology, 

n—>-+00 

lim in the sense of the uniform convergence, for all /c G N. 

n^+OD 


(90) 

(91) 


Proof. As mentioned above, the Twice Non-Degenerate Source Condition implies that 
Pt is a function admissible for Lemma 5, hence a certihcate for (Qo(yo))- As a 
result, ||gg ||2 ^ Ikrih and the sequence (gg)neN is bounded in L^(T). We may 
extract a subsequence qg' which weakly converges towards some q G LfiiT), and then 
||g ||2 ^ liminf„/^+oo ||<?o II 2 ^ Ikrih- Since <h* and are compact (see Lemma (6) in 
Appendix), we obtain that /ig = (<h*gQ')(^i converges toward p = <h*g for the (strong) 
topology of the uniform convergence. We immediately obtain that p{f) ^ 1 for all t G T, 
and p{xo^u) = 1, = 0 for all z/ G {1,..., A^}. 

Moreover, applying Lemma 3 to <h*gg (observing that x,, G Snl{r) fl S'n,l(r)), we 
get p^‘^\xo^v) = 0. As a result, q is admissible for (87), hence Hgrlh ^ ll^lh- Thus in fact 
Ikrlh = ll^lh and qr = q. Since the limit of the extracted subsequence does not depend 
on the choice of the subsequence, in fact the whole sequence converges. Moreover, the 
convergence is strong in L^(T) since lim„^+oo Iko II2 = Ikrlh- n 

As a consequence of the above convergence result, the third derivative precertihcate 
controls the extended support on thin grids. 

Proposition 15. Let mo G A1(T) (with {xoq,... a;o,Ar} C Qn) such that the Twice 
Non Degenerate Source Condition holds. Then, for n large enough, mo is a solution 
to (Qg(|/o)) and its extended support is given by: 

N N 

ext|,'')(mo) = IJ 5i’;^(r), and exti^)(mo) = (J (92) 

i/=i i/=i 


where 
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• Snl{r) is equal to {xo^v} or {xo,u - hn,xo^u}, 

• is equal to {xo^^y} or {xo^^,Xo^i, + hn}. 

Moreover, one cannot have simultaneously Snl{r) = — hn,Xo^,y} and Sn]y{r) = 

{XQ^I,, XQ iy -|- hfi} ■ 

Proof. By Lemma 5, tuq is a solution to (Qo(|/o)) and fj,T is a solution to (£^q (|/o))- 
Applying Lemma 3 to /ig, /xt, we see that Sn}u{r) is of the form 0, {ihn} or 
{(i — l)hn, ihn}, and that S'n,t(r) is of the form 0, {jhn} or {jhn, {j + with i ^ j. 

On the other hand, by the extremality relations between /Xg (solution of (^^g (|/o))) and 
rrig (solution of (Qg(|/o))), xq^u G Sn}u{r) and xq^^ G S'i'l(r). As a consequence Snl{r) 
is equal to {xo^^} or {xq^^ — hn,XQ^,y}, and S'i';l(r) is equal to {xQ^iy} or Xg^i, + hn}- 
Now, since /ij’(O) ^ 0, the fourth point of Lemma 3 ensures that for n large 
enough, one cannot have simultaneously Snl{r) = {xq^^ — hn,XQ^y} and Sn^u{r) = 

{Xo,u,Xo,n + hn}. □ 

Remark 12. As Proposition 15 shows, for each original spike, at most one pair of spikes 
appears at low noise : the original spike slightly shifted and either the immediate left 
neighbor shifted by +h„/2 or the immediate right neighbor shifted by —h„/2. 

We are now in position to provide a sufficient condition for the spikes to appear in 
pair, with a prediction on the location of the neighbor. 

Theorem 4. Assume that the operator "h® j has full rank and that the 

Twice Non Degenerate Source condition (Definiton 8) holds. Moreover, assume that all 
the components of the natural shift 

p = (4<f ntgytifrj* (93) 

are nonzero. Then, for n large enough, and all u ^ {1,..., N}, 

If p,y > 0, then S}fl{r) = {xg,^. - hn,Xo^,y}, and = {^o,u}, (94) 

If Pu < 0, then S}}l{r) = {xo,u}, and S'9^(r) = {xo,u,xo,u + h„}, (95) 

so that the extended support of mg on the grid Qn has the form 

ext)^"’^(mg) = {xg,!,..., Xq^n} U {xo^n - K ] z/ G |1, iV] and p^ > 0} 
ext^9^ I y + hn ; iz e {1, Nj and p,. < 0} . 

Corollary 2. Under the hypotheses of Theorem 4, for n large enough, there exists 
constants Cn'^ > 0, Cn^ > 0 such that for X ^ mini,g,^^Ar |Q;g^,^|, and for all 

w G L^(T) such that ||tc ||2 ^ Cn'^X, the solution to {Qf{y)) is unique, and reads 

= T.u=Mx,Jxo,,y+u + /3\,Jxo,.+6,yhJ, where 

—hn/2 < tn < hn/2 and e =' — 


sign (p). 
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Proof of Theorem 4- We proceed as in the proof of Theorem 2 by building a 
good candidate for /ig and using Lemma 2. To comply with the notations of 
Lemma 2, let us write Ylu=i ^o,idxo,^ = and =' =' I =' 

{i e |0, Pn - Ij ; ao,j 7^ 0}. 

For any choice of shift (ei)ie/ G {— we set =' U 

{i + Ei ] i E I and e* = —1} and =' i E I and e* = +1}. Since 

1^0,u — XQy\ > 2hn for o' ^ u and n large enough, we have Card + Card = 
3 X Card / = 3N. We shall hnd a choice of e such that Uj > 0 for all j E \ I, and 
Vj > 0 for all j E \ J, where 


uji" 1 a., 0>.. 


, A=M(^+|^)j(o {A-Ib) 


jm 


A = 4>c and B = . 


In this particular case where = j(i) = ah j \ /) U \ I) may be 

uniquely written as j = i + Sj for some i E I, where Ei E {—We may swap the 

columns of Ah so as to reformulate the condition ' 

( Ul\ 


vjm 


= -{AlAh) I 3 V into 


vj 

\irj 






/ lAf\ 
Iat 
\tNj 


where Ah =' + ^Bj diag(e) Ai - ^Bj diag(e) Aj+e - -^-Bz+g diag(e) j and U > 

0 for all i E I. But a Taylor expansion yields 

Ai+s - Y^i+ediagiE) = diag(e) +(hn)^73$i? diag(e) + o{hJ), 


=A 


where we dehned 



kl (k-1)1x2' 

Hence, we may apply Lemma 9 to 73 *^®? cliag(e) so as to obtain 


(96) 


tl = 


73 


■ diag(e)p + o 


hri 


Therefore it is sufficient to choose e = — sign(p) to make all the components of ti 
nonnegative. 

With that choice of £, it remains to prove that 


max 


(-4 + 

(4 - 


Ah 


Uj(r) 

VjO) 


< 1 . 
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Let us write =' Ah ( ) • Since [ ) = -{AlAh) ^Isiv, we get 

\^jm J \^jm J 

and applying Lemma 9 to diag(e) and 73 $® diag(e), we see that g„ converges 

towards qr (using ( 88 )). 

By construction of g„, 

Vj e JW \ I, (4>*g„ + ^(4.*g,)')(jh„) = 1, 
and Vj e J(‘) \ I, ($*g„ - ^(<|.*g„)')(jh„) = 1, (97) 

which may be summarized as 


Vi e I, (<h*g„ - Siy (<h*g„)')((i + £i)h„) = 1. 

Arguing as in the proof of point (iv) in Lemma 3 (replacing “1 = ..with 
“1 ^ ...” and using that pLp\xo^u) > 0 ), we may prove that for n large enough, 
($*g„ + ei^{^*qn)'){{i - ei)hn) < 1. 

Then, by the same argument of compactness and local concavity as in point (ii) of 
Lemma 3, we observe that 


|a; e |0, Pn-Ij; ($*g„ + y(4>*g.)')(A:h,,) ^ l| C J^, 
|a; e |0, Pn - 11 ; i^*qn - Yi^*qny)ikhn) ^ l| C 


max 


inclusions 

h 


and those 

'(-4 + 

,(^ - |B)7. 

Hence, by Lemma 2, c 


are m 


fact equalities. That precisely means that 


< 1 . 


*qn is the minimal norm certificate /Tq and {J^^^hn, J^^^hn) is 


the extended support. This concludes the proof. 


□ 


5.6. Asymptotics of the constants 

Again, we may examine the asymptotic behavior of the constants given in 
Corollary 2. Those constants stem from Theorem 3 which is itself a variant of Theorem 1 
for the Lasso. 

Replacing the constants Ci,..., C 3 of the proof of Theorem 1 with the corresponding 
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expressions for the C-BP, and using Lemma 9 we get 


Cl,n — II 


0 


C2,n — 


C3,n — ( ||-R(j(0\/(r))u(j(>)\/('))'^ft l|oo,2 


y-ninty - ||(43),.nsg))-i4g).-n||_^^ 

where jk is dehned in (96). As for C 4 ^n and cs^, like in the case of the Lasso, their 
expression lead to a pessimistic bound for the low noise regime, and we are led to make 
hner majorizations. 

Proposition 16. The constants Cn \ Cn'^ in Corollary 2 can he chosen as Cn^ = 0{hn^) 
and Cn'^ = 0(1), and one has 


nun* -pi 


( 100 ) 


w A 

I Vn. I On. 


( 101 ) 


Proof. The proof of (101) follows from (98) and (99). Using the reformulation {Qx{y)) 
of the C-BP as a (positive) Lasso, we have to ensure that (23) holds, or more precisely. 


y — Aa — Bb]\ < A 


(A* -I- 

(V - ^B*), 


where A = <Pg^, B = Let Ai, = + n be the 

orthogonal projector onto ker A)) = (Im A/i)-*-, and u = Since 

y — Aa — Bb = w — Ah{A*fiAh) ^A\w + \Ah{A\Ah) = hire -|- XA^’*tsN, 


we are led to check that 

(cn + A/i^)(jh„) + + \pf,)\3hn) < A for all j G (J^)^, 

{u + \p,Q){jhn) - - Apo)'(j/i„) < A for all j G (/‘^)'^, 

where /Xq =' <h*(A,)).4./i)“^)l37v yields the minimal norm certiheate 


( 102 ) 

(103) 


(ho - ¥/^o)(^n) 


(A+frB)’ 

(A-|i?) = 


{A*f^Ah) ^l3A- 


Given 0 < r < A min,,^,,/ |a;o,t. - a;o,t.'|, let N{r) =’ - r,xo^u + r) he a 

neighborhood of the Xq^A^- By the Twice Non-Degenerate Source condition, we may 
choose r > 0, such that 


—ki = sup pf{t) < 0, and k 2 = inf pr-p \t) > 0. 

teAf(r) ieV(r) 
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By compactness, =' Hrit) < 1- 

Let us recall that /Tq —)■ pr in the sense of the uniform convergence (and similarly 
for the derivatives). As a result, for n G N large enough. 


ki 


k‘2 


sup (yUo)"(t) < < 0, inf > — > 0, sup < 


1 + ^3 


teN(r) 


t&N{r) 


teT:\N{r) 


< 1, (104) 


YII(A‘;)'=’'|Ioo « h and L||(^;)'||„ 
Now, we assume that is small enough, so that 

itnlb h 




1 — k-i 


||($W)^ 


< V. tor e {2.3}. ll(1><‘‘')‘IU,2hL < 


A 


4 ’ 


and ||(4><‘>)*|U,2h{!2 < ^ 


, for /c G {0,1}, 


(105) 


A ' 6 

Then, using the fact that and \u^^'>\(t) < ||(*hi^i)*||oo, 2 || tc ||2 and ^ 1, we obtain 


sup 

teT\7V(r) 


U 


A+trS + Y 




(y + To)] ) (^) < 1- 


Thus it remains to prove that for each z/G 

+ /^o + Y(y + Mo)'^ {t) < 1 for f G {xo,u - r, Xo,u + r)\ S^l{r), (106) 

and (y + - Y(y + To)'^ (^) < 1 for f G {xo,u - r, xo,u + r)\ 5'O^r). (107) 

We only deal with the case Snl{r) = {xq^^}, 5'n,l(r) = {xQ^t,,XQ^t, + h„}, the symmetric 
case being similar. Let / =' ^a;(- — Xq^) + Toi' ~ Hy dehnition of B, 

i^{xo,u) = (^'{xq^u) = {^({Toy + hn) - ^u{xo,u + hn) = 0, SO that 

/(0) = 1, /'(0) = 1, and /(h„) - ^/'(h„) = 1. (108) 

Moreover, from Eq. (104) to (105), and letting ki = k 2 = we deduce that 

VtG(-r,r), f"{t) + ^\f^^\t)\ <-ki <0, and /(^i(t) >/cs > 0, (109) 

so that the strict concavity of / — implies that (/ — < 1 for t G 

(-r, -hn) U (0,r). 

It remains to prove that (/ + < 1 for t G (—r,r) \ (—h„,0]. A Taylor 

expansion of / and /' yields (writing as usual 7fc = ^ — (^- 1 ) 1 x 2 ) 

1 - (/(fc) - y/(a»)) = 1 - /(O) - y{''('>> - KSif^Ko) + 

'-' "-V. -' 

=0 =0 

1 ~ {f{~hn) + —f\ — hn)) = 1 — /(O) + +h„^73/^^^(0) — h„^74/^"^^(0) + i?2(hn)- 

'-V -' 

=0 
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1 - ifi-hn) + y/'(-h„)) = -2h„S/^"nO) + {Ri + R 2 ){K) (110) 

^ 2(—74)A;2/i„^ + (-Ri +-R2)(hn), (111) 

where 

fli + = hj + /<^'(-sA,.)) (^1^ - ^1^) d®. (112) 

with ||7>|U!(||liJ®l|oo + ll(ti;)®l|oo = 0(l), (113) 

Hence, 

1 - ifi-hn) + y/'(-M) ^ 2(-74)A;2 hj + 0(h/) > 0. (114) 

>0 

Moreover,by the strict concavity of / + we also deduce that (/ + ^f')it) < 1 for 
t G (—r, —hn] U (0, r), thus we get the local inequalities (106) and (107), hence the global 
inequalities (102) and (103). 

To conclude, the constants in the condition on are 0(1), and gathering the 

i' 1 \ o (O') 

asympotics for ci^n, C 2 ,n, we obtain On = 0(hn ), On = 0(1). □ 

6. Numerical illustrations 

In this section, we illustrate the usefulness of our analysis to gain a precise 
understanding of the recovery performance of f'^-type methods (Lasso and C-BP) for 
both deconvolution and compressed sensing problems. The code to reproduce these 
numerical experiments is available online^. 

6.1. Convergenee of pre-eertificates 

In this section and in Section 6.2, we consider the deconvolution problems in the 
case where (p is an ideal hlter, i.e. whose Fourier coefficients 

VA; e Z, p{k) = [ </p(t)e-2‘-^*df 

satisfy (pik) = 1 if /c G {—/c, ...,/c} and ipik) = 0 otherwise. This allows us to 
implement exactly the <h operator appearing in the Lasso and C-BP problem since 
Im(<l>) is a hnite dimensional space of dimension Q = 2/c -7 1, i.e. it can be represented 
using a matrix of size iQ, P) when evaluated on a grid of P points. In Figures 1 and 2 
we used fc = 10. 


f https://github.com/gpeyre/2015-IP-lasso-cbp/ 
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N = 2 N = 3 

Figure 1: Display of rjy (red) and fix (blue) pre-certificate for different input positive 
measures mo (showed as black dots to symbolize the position of the Diracs). 

Figure 1 illustrates for the case of two (A^ = 2) and three (iV = 3) spikes 
the behavior of the vanishing pre-certihcate r]y (see Dehnition 3) useful to analyze 
Lasso/BLasso problems and of the pre-certihcate (see Dehnition 7) useful to 
analyze C-BP problems. 

We hrst notice that for all the (positive) input measures (i.e. whatever the 
spacing between the Diracs), rjy is always a non-degenerate certihcate (in the sense 
of Proposition 3), meaning that one actually has rjy = r]^ (where the minimal norm 
certihcate r]^ is dehned in (37)). This empirical hnding is the subject of another 
recent work on the asymptotic of sparse recovery of positive measures when the spacing 
between the Diracs tends to zero [16]. Since r]^ is non-degenerate, one can thus apply 
Theorem 2 to analyze the extended support of the Lasso (see below Section 6.2 for a 
numerical illustration). 

For the C-BP problem, the situation is however more contrasted. We observe 
that when the Dirac masses are separated enough (hrst row) then the pre-certihcate 
jiT is a valid certihcate, meaning the the Twice Non-Degenerate Source Condition (see 
Dehnition 8) holds. This means that Theorem 4 can be applied to analyze the extended 
support of C-BP (see Section 6.2 below for a numerical illustration). But when the 
Dirac masses are too close (second and third rows), one has ||/iT||oo > 1, so that one 
cannot ensure the support stability of the C-BP solution with our result. 

6.2. Extended support for deconvolution 

We still consider the case of an ideal low pass hlter. Figure 2 displays the evolution, 
as a function of A (in abscissa) of the solutions ax of and of {ax, bx) of {Qx{y))- 

We consider here the case of an input measure with two nearby Diracs (displayed as 
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red/blue dots in the upper-left part of the Fip-nrpl and whpn thprp is no nniRP i p m = n 
Each 1-D curve (either plain or dashed) : 
e.g. {a\)i, for some index i (only non-zen 



Figure 2: Display of the evolution as a function of A of the solutions of the Lasso 
and C-BP problems. Note that dashed curved have been (artihcially) slightly shifted to 
avoid that they overlap with the plain curve. 

The solutions path A i—)■ oa (for Lasso) and A i—)■ (oa, &a) (for C-BP) are continuous 
and piecewise affine, which is to be expected since the regularizations {G and G under 
conic constraints) are polyhedral. The upper-left plot in the hgure displays the pre- 
certihcate rjy (in magenta, see Dehnition 3) and pr (in green, see Dehnition 7). 
This shows graphically that these two precertihcates are non-degenerate (according to 
Dehnitions 2 and 8) so that the results of Theorems 2 and 4 hold, hence precisely 
describing the evolution of the solution on the extended support when A is small. On 
these graphs, this corresponds to the hrst segment of the corresponding piecewise affine 
paths. 

The behavior for BP agrees with our analysis. As predicted by Theorem 2, there 
exists a range of values 0 < A < Aq on which the solution is exactly supported on the 
extended support J, which is composed of four spikes (the plain curve corresponds to the 
support I and the dashed curve corresponds to J\I). Also, as predicted by Proposition 7 
in the case w = 0, we verify that Aq = 0{hn) and that the Lipschitz constant of A i—)■ oa 
is of order 0(l//i„). 
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In sharp contrast, the behavior for C-BP is less regular, since the range 0 < A < Aq 
on which the solution is supported on the extended support is shorter, as it can be clearly 
seen on the zoom for very small values of A. This is in agreement with Proposition 16 
which shows that Aq is of the order of 0{hn^) and that the Lipschitz constant of 
A I—)■ (oa, h\) is of order 0{l/hr^). On this range of small A, as predicted by Theorem 4, 
the support of the solutions (which correspond to the extended support J described 
in Theorem 4) is composed of one pair of neighboring spikes for each original spike. 
For indices on the support i E I, one has \{bx)i\/{a\)i < h/2 (the constraint is non¬ 
saturating, and the spike moves “freely” inside {ih — ih -|- |)) while for indices on the 
extended part i G J\/, one has \{bx)i\/{ax)i = h/2 (the constraint is saturating, the 
spikes are hxed at half-grid points). Another part of the path is interesting, for A not so 
small (say A > Ai), which is in fact the prominent regime in the non-zoomed hgure. For 
this range of A, there is still a pair of spikes for each original spike, but this time both 
spikes saturate, on same side. This observation should be related to Proposition 13 and 
Remark 10 which predict that, in the case where 7^ 0, the C-BP yields either 

one spike or a pair of spikes with the same shift (the latter case is in fact overwhelming). 

6.3. Extended support for compressed sensing 

To show the usefulness of our “abstract” support analysis of the Lasso problem 
(Section 4), we illustrate its use to analyze the performance of G recovery in a 
compressed sensing setup. Compressed sensing corresponds to the recovery of a high 
dimensional (but hopefully sparse) vector oq G from low resolution, possibly noisy, 
randomized observations y = Bqq + w E see for instance [8] for an overview of the 
literature on this topic. For simplicity, we assume that there is no noise {w = 0) and 
we consider here the case where B E is a realization from the Gaussian matrix 

ensemble, where the entries are independent and uniformly distributed according to a 
Gaussian A/’(0,1) distribution. This setting is particularly well documented, and it has 
been shown, assuming that oq is s-sparse (meaning that | supp(ao)| = s), that there are 
roughly three regimes: 

— If s < So =’ 2iog(P) ’ then ao is with “high probability” the unique solution of {Vo{yo)) 
(it is identihable), and the support is stable to small noise, because rjp (as dehned 
in (27)) is a valid certihcate, ||^7 f||oo ^ 1- This is shown for instance in [36, 20]. 

— If s < Si =' 2 log^/Q) ’ then oq is with “high probability” the unique solution 
of (Vo{yo)), but the support is not stable, meaning that yp is not a valid certihcate. 
This phenomena is precisely analyzed in [11, 1] using tools from random matrix theory 
and so-called Gaussian width computations. 

— If s > si, then oq with “high probability” is not the solution of {Vo{yo)). 

We do not want to give details here on the precise meaning of with “high probability”, 
but this can be precisely quantihed in term of probability of success (with respect to 
the random draw of B) and one can show that a phase transition occurs, meaning that 
for large (P, Q) the transition between these regimes is sharp. 
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While the regime s < Sq is easy to understand, a precise analysis of the intermediate 
regime Sq < s < Si in term of support stability is still lacking. Figure 3 shows how 
Theorem 1 allows us to compute numerically the size of the recovered support, hence 
providing a quantification of the degree of “instability” of the support when a small noise 
w contaminates the observations. The simulation is done with {P,Q) = (400,100). 

The left cuve shows, as a function of s (in abscissa), the probability (with respect 
to a random draw of <h and Oq a s-sparse vector) of the event that Oq is identihable 
(plain curve) and of the event that rjp is a valid certificate (dashed curve). This clearly 
highlights the phase transition phenomena between the three different regimes, and one 
roughly gets that sq ~ 6 and si ~ 20, which is consistent with the theoretical asymptotic 
bounds found in the literature. 

The right part of the hgure, shows, for three different sparsity levels s G {14,16,18}, 
the histogram of the repartition of | J| where J is the extended support, as dehned in 
Theorem 1. According to Theorem 1, this histogram thus shows the repartition of 
the sizes of the supports of the solutions to (Vx{y)) when the noise w contaminating 
the observations y = Bao + w is small and A is chosen in accordance to the noise 
level. As one could expect, this histogram is more and more concentrated around the 
minimum possible value s (since we are in the regime s < Si so that the support I of 
size s is included in the extended support J) as s approaches Sq (for smaller values, 
the histogram being only concentrated at s since J = I and the support is stable). 
Analyzing theoretically this numerical observation is an interesting avenue for future 
work that would help to better understand the performance of compressed sensing. 

Conclusion 

In this work, we have provided a precise analysis of the properties of the solution 
path of f'^-type variational problems in the low-noise regime. This includes in particular 
the Lasso and the C-BP problems. A particular attention has been paid to the support 
set of this path, which in general cannot be expected to match the one of the sought 
after solution. Two striking examples support the relevance of this approach. For 
the deconvolution problem, we showed theoretically that in general this support is not 
stable, and we were able to derive in closed form the solution of the “extended support” 
that is twice larger, but is stable. In the compressed sensing scenario (i.e. when the 
operator of the inverse problem is random), we showed numerically how to leverage our 
theoretical hndings and analyze the growth of the extended support size as the number of 
measurements diminishes. This analysis opens the doors for many new developments to 
better understand this extended support, both for deterministic operators (e.g. Radon 
transform in medical imaging) and random ones. 
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Figure 3: Left: probability as a function of s of the event that Oq is identihable (plain 
curve) and of the even that its support is stable (dashed curve). Right: for several value 
of s, display of histogram of repartition of the sizes \J\ of the extended support J. 
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Appendix A. Useful properties of the integral transform 

Lemma 6. Let K G W and assume that tp G C^iT x T). Then for all p G T^(T), 
^*p G C'^(T) and for alike {0,1... K} 


\/y G T, {^*pf^\y) = / {d 2 f^{x,y)p{x)dx. 

Ji 


(A.l) 


Moreover, the adjoint operator 


p 


C{T) 

j'f{d2fT{x,y)p{x)dx ’ 


(A.2) 


is compact for all k e {0,1,... K}. 

Proof. The hrst part of the lemma is a standard application of the Lebesgue dominated 
convergence theorem. 
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As for the second part, it is a consequence of the Ascoli-Arzela theorem. Indeed, 
let 5 ^ 2 ( 0 ,1) = {p G h^(T); ||p ||2 ^ 1}, and kF = p G i?L2(0,1)}. Then kF C C(T) 

is bounded since 

^ ^ {{d 2 )’^(p{x, y)f dx^ J^{p{x)ydx ^ V|| (^s) VIU 

Moreover, it is equicontinuous since 

= [{{d 2 )’"‘p{x,y) - {d 2 )’'<p{x',y))p{x) (A.3) 

Jt 

^ (A.4) 

where (X(^g^)k^ is the modulus of continuity of (< 92 )^ 93 . Thus Ascoli-Arzela’s theorem 
ensures that 1) is relatively compact, hence the result. □ 

An interesting consequence of the above lemma is the following. Given any bounded 
sequence {pnjneN in -h^(T), we may extract a subsequence {pn'jn'eN which converges 
weakly towards some p G L^(T). Then, the (sub)sequence <h*Pn' converges towards <h*p 
for the (strong) uniform topology, and its derivatives also converge towards 

^(k),*P for that topology. 

Appendix B. Asymptotic expansion of the inverse of a Gram matrix 

In this Appendix, we gather some useful lemmas on the asymptotic behavior of 
inverse Gram matrices. 

Lemma 7. Let A: — )■ T^(T), B : — )■ be linear operators such that A has full 

rank and B is invertible. Then {AB)~^ = B~^A^. 

Proof. It is sufficient to write 

{{AB)*{AB))~^ {AB)*) = B-\A*A)-^B-^’*B*A* = 5"^+. 

□ 

Lemma 8. Let A, B, B^-. — )■ L^(T) be linear operators such that B^ = B + 0{h) for 

h > 0, and that (^A B^ has full rank. Let 11 be the orthogonal projector onto (Im A)-*-, 
and let 
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A A hBh ) — — 


{B*YiB)-^B*A+^*s 

-{B*Y{B)-^B*A+^*s 

{B*Y{B)-^B*I{ A 
-{B*UB)-^B*UI ^ 


+ 0 ( 1 ), 


0 ( 1 ), 


but [A A + hBh 


= - UB{B*UB)-^B*A+’*s + 0{h). 


(B.l) 


(B.2) 


(B.3) 


Proof. Observe that (^A A + hBh^ = (^A Bh^ 


In In 
0 hl]\ 


so that 


Gh = 


In 0 \ (A*A A*Bh\ In In 


In hiN) \ Bh^ B'^Bh / \ 0 hi, 


Since [A B] has full rank, the middle matrix is invertible for h small enough, and 


^-1 _ (In ~pIn\ IA*A A*Bh 
“ I 0 \In ) [B*hA BlBh 


It It 
— tIn tJ-n 


Writing 


a d^f. MM A*Bh 

c d ~ [bIA BlBh 


, the block inversion formula yields 


a M a ^bS ^ca ^ —a ^bS ^ 
-S-^ca-^ S-^ 


where S = d- ca-^b = BlBh - BIA{A*A)-^A*Bh = BlYiBh 

is indeed invertible for small h since {^A B^ has full rank. Moreover, a~^bS~^ = 
A^Bh{BlUBh)-\ and P-^ca-^ = (BlUBh)-^BIAS’S 

, ^-1 I s\ I In —tIn\ I a~^s + a~^bS~^ca~^s\ 

Now, we evaluate G. = • We obtain 

\s I \ 0 fiN I \ -S '-ca Is I 


. We obtain 


1 ( S-^caSs \ if (B*UB)-^B*A+’*s \ 


Eventually, by Lemma 7, [A A + hBh ) = 


We obtain 


A A hBji I — 


In ~pIn\ I A*A A*Bh 
0 Un ) \BIA BlBh 


Un\ M+ - A-^Bh{BlYlBh)-^BlIl 


0 fh 


iBlYlBh)-^BlYl 
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(^A A + hBh^ 


if 


and i^A A + hBj^ 


- YiBH{BHliBh)-^BlA+^* YiBHiBlIlBh)-^') 



= yl+’*s - n5(5*n5)-^5M+’*s + 0{h). 


□ 


Lemma 9. Let A, B,C,Ch'- —)■ L‘^(T) be linear operators such that Ch = C + o(l) 
for h > 0, and that {^A B cj has full rank. Let H be the orthogonal projector onto 

(Im (^A B^)-^, and let 


Cj/i — 


{A + \By \ 
{A-\BY 
{A + \B + h^Ch)*) 


{a + \b a-^b 


A + \B + h^Ch) . 


Then for h > 0 small enough, Gh and C*YiC are invertible, and 

f-lN\ 




Iat 

{InJ 


1 


0 

\In / 


{C*UC)-^C*{A B 



+ o 





(^A+^B A-l^B A + llB + h^ChY 


but 


0 

^ {C*IIC)-^C*U y 





/ A* + IB* ^ 

+ 

( 

A* - 


In 

j 


[tMj 




Proof. Observe that 


+ a + = (a B 



In 

—In 

0 


In'^ 
In 
In y 
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As a result, for h > 0 small enough Gh is invertible and 

-IA , 2 i \ 

Gh'= An -An 0 diagfl,-, —J B*A B*B B*Ch 
\o 0 J ^ ^ \CIA CIB CIC,J 

An An 0 
An —An 0 
-In 0 In 



the middle matrix being invertible from the full rank assumption on {^A B C^. 


Moreover, writing F A | ^4 B] and 


r*r T*Ch 
cA etc. 


, we obtain 


A*A A*B A*Ch 
B*A B*B B*Ch 
CIA CIB etc. 


-S ^ca~ 


-a-^hS-^ 

s-^ 


where u =' a ^ + a ^bS ^ca S =' d — ca = CtAC,, a ^bS ^ = 

(Y*t)-^T*Ch{CttlCh)-\ S-^ca-^ = {CtflChACtT{T*t)~\ and fl is the orthogonal 
projector onto (ImF)-*". Thus 
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Eventually, one has 

^A + ^B A-^B A + !lB + h^Ch^ 


(\In \In —In\ 


V 

1 

lA 

1 

lA 


\In —\In 0 
0 0 / 


diag(l,^,^) (r Ch 


N ) 

/o 0 —In\ 

0 0 0 
0 0 /tv 

\ 

0 


J 


u 

S~^ca~^ 


-a-^bS-^ 

s-^ 




r- c;l+o(^ 


and 


A + l^B A-i^B A + \B + h^Ch^ 



^ liv’ 
Iat 


VIat > 


V + 

\ 2 1 

(\In 

1 T 

2^N 

0 \ 




It 1 r 

2 J-N ~2^N 

—In 


Itv 

K 0 

0 

In j 


[tNj 


Q 

f+’* - fic'T,(C'^nc'T,)-^c';;f+’* 

Itv 


Iat 


Iat 


= f+’* * V) ~ nc'(C'*nc')-ic'T+’* I "q" 1 + o(i) 


□ 
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